AI/ML+Physics Part 3: Designing an Architecture

These are related to another perspective that promotes physicality and models. A great Einstein quote states that everything should be made as simple as possible to describe data but not simpler. In the era of machine learning, we seek models that are as simple as possible to describe data and no simpler. This principle of simplicity or parsimony has been the gold standard in physics for 2000 years. From Aristotle to Einstein, models that are more beautiful, parsimonious, and as simple as possible typically encapsulate the core bits of physics. These models are more interpretable and tend to generalize well without overfitting.

In the history of science, from astrology to astronomy, and from alchemy to chemistry, every major leap forward in our understanding of physics has resulted in simpler and more universal descriptions. This is a crucial point to consider. Another area where essential physics can be captured and discovered through machine learning is in the concepts of symmetries, invariances, and conservation laws. Most of our partial differential equations, such as mass conservation, momentum conservation, and energy conservation, typically arise from the conservation of some quantity.