Machine Learning for Scientific Discovery, with Examples in Fluid Mechanics

Reduced-order modeling is a technique used to simplify complex partial differential equations (PDEs) in spatial-temporal systems. This allows us to create a low-dimensional model tailored to a particular geometry or configuration. We will illustrate how this works using the flow past a cylinder, which has an interesting history.

In particular, Bear Noack and his colleagues, about 20 years ago, showed that a complex spatial-temporal system with around 20,000 degrees of freedom could be approximated with just three orthogonal modes: p o D one, p o D two, and the shift mode. When these modes are projected onto the Navier-Stokes equations, we obtain the differential equation, which is one of the models Noack introduced to describe how a quadratic nonlinear system can give rise to a cubic hopf bifurcation due to the separation of time scales and a slow manifold.

We decided to try this example using SINDy, but instead of using the Navier-Stokes equation, we only had access to measurement data. We projected this data onto the three modes, producing time series of X, Y and Z. We then used regression to learn a differential equation in X, Y and Z. The model we learned was consistent with the blue model and our identified system had the same slow manifold and parabolic attractor.

My colleague John Christophe Lazo made two incredibly important innovations to the SINDy framework that I believe are still some of the best innovations to this day. The first one was that when we are trying to build reduced-order models of fluid systems, such as flow past a cylinder, we already have a lot of knowledge about the physics. We know about energy conservation, skew symmetry, and the quadratic nonlinear charities. This allowed J.C. to write these items down as equality constraints on the coefficients of the SINDy model, and add them as a constrained sparse regression. This small addition of two lines of code made the model much more stable, as it conserved energy.

The second key innovation J.C. brought was allowing higher order terms to the model. He realized that by allowing cubic, quintic, and higher order terms, they could approximate the dynamic effect of the low energy modes that had been truncated, but were still important for the dynamics of the high energy modes that were being modeled. This included the dominant POD modes, as well as the first harmonic frequency modes, which had 96% of the energy.