Active Manifold and Model Order Reduction to Accelerate Multidisciplinary Analysis and Optimization

Hello everyone, welcome to the AIAA journal seminar series. I am Tom Xi, professor of Aeronautics and Astronautics at P University and editor in chief of the AIAA journal. Thank you for joining us. The seminar series was created to acknowledge our authors, reviewers, readers, and editors. It consists of keynote and author seminars. Keynotes cover emerging topics in aerospace to inspire research for publication in the AIAA journal. Author seminars discuss articles already published in the journal to expand on their significance. Today, we are pleased to have Charbel Farhat as our speaker for the author seminar. Before introducing Charbel, I want to thank the seminar series committee for organizing the events, as well as the AIAA and AIAA Publications Committee for their support. Authors are selected based on nominations from editors and board members, with the top five invited to speak. We encourage attendees to nominate authors for future seminars. Charbel Farhat is a renowned professor with numerous contributions to aerospace. He will discuss Active Manifold, a technique for efficient model order reduction in MDAO problems. The talk will include a tutorial on model order reduction as the foundation for the discussion. Thank you, Charbel, for sharing your insights. Feel free to ask questions in the chat during the Q&A session. Thank you.

Forming a class of many-query problems, PDE-based optimization requires frequent calls to partial differential equations or their solutions during parameter updates, making it computationally intensive. The multidisciplinary nature of these problems can further increase computational demands, especially in aerodynamics with viscous flows. To address these challenges, many turn to Surrogate Modeling.

In Surrogate Modeling, there are two main approaches: external representation and internal representation. External representation focuses on understanding how the output varies based on the input without delving deeply into the system. This approach includes techniques like response surface models, artificial neural networks, and Gaussian processes, which are popular for data regression due to their availability and ease of use. However, external representation may have limitations for vector quantities of interest and may not provide a full understanding of the system's physics.

In contrast, internal representation involves modeling the system by incorporating partial differential equations and understanding the physics behind the system. This approach can include simplified physics models or projection-based reduced-order models for surrogate modeling when detailed models are not feasible.

Two examples will be focused on to provide a spectrum of possibilities. Starting with the disadvantages, these methods are not as popular due to the high entry bar. A deep understanding of physics and computational modeling is required, along with awareness of limitations. One cannot simply download software for model reductions or simplified physics and expect immediate results. However, research is about pushing boundaries and advancing technology for future improvements.

On the other hand, the advantages are significant. By building a surrogate model of the system rather than the output, one can explore numerous quantities of interest without prior knowledge. This allows for easy exploration of parameter domains, especially in parametric applications like NBO. Spatial-temporal fields, including vector fields, can be effectively analyzed using this approach.

To illustrate, consider the example of parachute designs for landing Perseverance on Mars. Despite extensive testing, both the disc sail parachute and the ring sail parachute failed at lower forces than expected. This discrepancy may be attributed to the fact that the parachutes were tested in a subsonic wind tunnel, highlighting the importance of understanding the limitations of testing environments.

The community discovered a new failure mode related to supersonic parachute deployment, as opposed to subsonic deployment. This led JP L to develop a computational model for parachute opening in a flow regime. The model addresses challenges such as shocks, wakes, turbulence, and fluid-structure interaction with topology changes. It accurately reproduces flight test data for the Perseverance NASA Aspi program. The next step was to investigate parachute failure modes. When asked about the program manager's quantity of interest, the response was that if they knew the answer, they wouldn't need a computational model.

When designing aerodynamic systems, there is more to consider than just lift and drag. Failure modes for new systems can be unknown, making it difficult to focus on specific areas. Surrogate Modeling involves internal and external approaches, sometimes requiring both. Currently, we are working on a Multidisciplinary Design Analysis and Optimization (MDAO) for hypersonic systems, particularly during a pull up Maneuver. This involves designing the system geometry and trajectory to optimize performance.

The trajectory for the system includes various phases such as deep Dive, pull up, constant glide, and Terminal Dive. Constraints, labeled as C1, C2, etc., are imposed on the Objective function to minimize the time to land at a specific Distance. Different constraints may require different modeling approaches, such as Projection Based Reduced Order Models (PROMs) or neural networks.

For instance, the Objective function, which aims to minimize the time to land at a point, can be effectively modeled using neural networks or response surface models. Constraints like maintaining laminar flow or setting temperature limits may require PROMs for accurate representation. Each constraint may call for a specific modeling approach to ensure optimal performance of the aerodynamic system.

Constraint C7 pertains to geometric and box constraints, making any surrogate model suitable for the task. The challenge lies in integrating various technologies to work together, with a focus on training. Training a surrogate model is costly due to the curse of dimensionality, regardless of the approach taken. This curse refers to the dimensionality of the parameter domain, not the model itself. Regression-based approaches offer some advantage in execution, while Proper Orthogonal Decomposition (PR) models excel in accuracy with comparable training data. The primary concern addressed in this presentation is the curse of dimensionality during training. Reduced Order Models are proposed as a technique to mitigate this challenge.

Reduced Order Models are a specific technique that can be applied to any type of representation, such as regression-based or model-based. The Basic Version involves Projection Based Model Order Reduction (PMOR), which is a method that results in a projection-based reduced-order model. The emphasis on projection is important because it provides a clear understanding of starting from a high-fidelity model and projecting it onto a lower-dimensional subspace. This technique is specific and has a community dedicated to it.

In the modern age of data and machine learning, it is crucial to recognize that physics-based machine learning methods are essential for model reduction. The process begins with a semi-discrete or discrete computational model with parameters. The goal is to approximate the solution accurately in a lower-dimensional space. This involves creating a hypothesis, building a representation, and using data-driven learning to construct a reduced basis. The final step is to minimize a loss function by finding generalized coordinates that minimize the residual.

Overall, the process of building a projection-based reduced-order model incorporates elements of machine learning, including hypothesis formulation, representation building, data-driven learning, and loss function minimization.

One approach we have proposed in the past to deal with this issue is the concept of a quadrature rule in hyperspace. This involves computing a projection on selected elements of a mesh, rather than the entire mesh, in order to speed up the process. Similar to a Gauss rule in calculus, only a few points are needed to evaluate the function at those points. The weights for the selected elements must be positive and determined using machine learning techniques based on solution snapshots. Once training is complete, the original projection is transformed into a hyperreduced projection, providing the desired advantages.

For example, in a CFD application using a large-scale mesh over the NASA Common Research Model, hyperreduction focuses on a small set of elements, as shown in color.

I am only showing the elements that are on the surface. In general, we go from 11,454,702 cells to 5,000 cells, which is less than 0.01% of the total elements, but still provides the necessary accuracy. This process is part of Structural Dynamics. The image on the left shows the full mesh, while the image on the right displays only a few elements in blue that were used for computation. The size of the finite element mesh decreased from 275,685 to 859 in order to accurately compute these projections.

The State of the art PMOR can handle highly complex problems, such as aerodynamics. An example is an F-16 maneuvering at a Reynolds number of 18 million. The high dimensional model, a detached simulation with a dimensionality of 160 million, has been reduced to 54 dimensions. Despite this reduction, the flow field looks the same whether using the high dimensional model or the hyper-reduced order model. This can be seen in the comparison of vortex shedding on the left and right sides.

The drag coefficient is indistinguishable between them, indicating high accuracy. This results in a significant speedup from 100 hours on 3584 cores to less than six minutes on 32 cores. This speedup can be leveraged within MDAO to accelerate processes. An example from Structures demonstrates the evolution of technology, moving beyond flat plates and beams to complex crash analysis of the front bumper of a 2013 Honda Accord EX-L. The finite element model, developed with LS-DYNA, undergoes explicit transient dynamic analysis. The model's dimensionality has been reduced from 10 million to 107, with the mesh reduced from 2 million elements to 2000 elements. This reduction results in a decrease in CPU time from 40 hours on 250 cores to eight seconds on one core, showcasing significant speed improvements.

The results from the RAM show the crash of the front bumper, which is accurate. This is a convergence study analysis. A RAM with a dimension of 20 would not be sufficient, but a RAM with a dimension of 100 would accurately reproduce the main features of the structure's response.

The benefits of Learning with models and data versus Learning with Data Only are significant. When you do Data regression, you are Learning only with data. With RS, you are Learning with the models because you start from the physics and with data because we compute these snapshots.

For example, consider the problem of a Mirage F one and the sloshing of a fuel tank. There is a couple fluid-structure interaction, with two different systems of fluid - the aerodynamic from the outside and the sloshing of the fuel from the inside. We are interested in the Flutter Speed Index.

About 10 years ago, we demonstrated that this concept could be implemented on the first iPhone. We were able to show fluid-structure interaction grounded in CFD.

When analyzing the Flutter Speed Index on an iPhone, two parameters are considered: Fill level and Mach number. The Ground truth data is on the left, but this information is not supposed to be known. This creates a verification problem. The database on the right captures some aspects of the French dip, but it does not include a secondary dip around Mach 1.1. This missing data is represented by a circle.

Using Data regression, the database is smoothed and points are filled to approximate the Flutter Speed Index accurately. However, the ground truth data cannot discover the secondary dip because it is not in the database. Extrapolation becomes tricky when trying to go outside the known data.

By using regression with a model instead of the data, the French dip can be captured even more accurately. Additionally, the model can discover the secondary dip because it is based on the models being worked with.

Data and models can reproduce depth even when it is not in the database, which is the main highlight. The main concern of this topic is the curse of dimensionality, and the goal is to mitigate it rather than completely eliminate it. Sampling methods for training surrogate models can be classified into non-adaptive and adaptive approaches.

Non-adaptive sampling, such as uniform sampling, is one extreme approach that some may consider efficient, but it can be costly depending on the application. Uniform sampling is not ideal for high-dimensional spaces, as it can be challenging to work with unless the dimensionality is lower than five for CFD and ten for structures. Latin hypercube sampling is another popular method but may struggle with higher dimensional spaces.

In contrast, adaptive approaches, such as sequential design of experiments, active learning, and iterative sampling, offer more flexibility. With adaptive sampling, feedback is obtained after each sample, allowing for targeted sampling in areas that require further resolution. This approach is particularly useful in hardware testing and machine learning.

Uniform sampling illustrates the curse of dimensionality that affects both approaches. Adaptive sampling is preferred because it mitigates the curse, although it is still present. To make the point, consider using uniform sampling with at least two points in each dimension. In a parameter space of dimension ν, two to the power of ν samples are needed for effective training. For example, with a dimension of five, 32 samples are required, and for a dimension of 10, over 1,000 samples are needed. In shape optimization, dimensions of 15 to 20 are common, requiring millions of samples for training a reduced order model.

If you are conducting a data regression, you will need more data than usual. In the case of the runs, the physics can provide some help, but it also highlights the curse of dimensionality.

A standard procedure today combines a sequential design of experiments with active learning and iterative sampling. This procedure is guided by an understanding of the residual, which serves as an indicator for most applications. Each time a new point is sampled, the training accuracy achieved so far is taken into account with respect to previous points.

The procedure can be used to build a global reduced order basis that is accurate across the entire parameter domain, or to build a database of local reduced order bases and models that are trained at specific parameter points but can be interpolated between. Interpolation on a matrix manifold is used to preserve the structure and constraints of what is being interpolated.

In the generic case, a two-dimensional parameter space with components mu one and mu two is considered. An initial point is selected to build a RAM or reduced order basis, followed by iterations where random candidate points are generated and evaluated using the initial RAM. The point with the highest residual is then selected based on the residual indicator.

If a point has the highest residual, it means that the current model is not accurate at that point. To improve accuracy, that point should be added to the sample points. Retrain on the snapshots from these two points and repeat this process until a non-uniform sampling is achieved. This adaptive method considers feedback and helps reduce the curse of dimensionality, although it does not completely eliminate it.

The main focus of this paper is to apply the concept to solve an MDAO problem. The governing PDEs are separated into linear and nonlinear equations. The applications will demonstrate the multidisciplinary nature of the problem. The RL represents the residual for linear PDEs and RNL for heavily nonlinear CFD-based turbulent flow. The solutions are Q, Z, and μ is the parameter vector. An MDAO approach will be used to reformulate everything in terms of a function of μ. The constraints, such as Box Constraints, are scalar and not necessarily PDEs. The idea is to create a database of local linear ROMs for flutter.

For example, flutter is essentially a linearized problem. In addition, a global nonlinear ROM would need to be built on the aerodynamic side to predict lift and drag, even though data regression could also be used for this purpose. Despite this, there are advantages to using an internal approach. It has been observed that the number of points required for training a ROB/ROM increases significantly with the dimension Nμ of the design parameter domain.

The number of points needed for training a ROB/PROM increases with the dimension of the design parameter domain, Nμ. The idea is to train only in the neighborhood of the trajectory path instead of the entire parameter domain to save time. The challenge is to determine where the trajectory will lie in the space. By anticipating the trajectory path, the optimization problem can be formulated in a subregion of the parameter domain, denoted as D_AM. Training is then done only in this subregion, which is more efficient due to lower dimensionality.

Training is associated with exercising a high-dimensional model, leading to training in a lower-dimensional nonlinear solution manifold called the inactive manifold. The goal is to discover this manifold to save on computational costs. By investing offline in solving a less computationally intensive MDAO problem related to the original one, the active manifold can be discovered. Training is then done in the active manifold, leading to significant savings in computational costs and accelerating the solution of the MDAO problem.

Over the past five years, improvements have been made in ensuring the robustness of the Active manifold to contain the trajectory of optimization. However, the width of the Active manifold remains unknown. The focus will be on two problems: one involving a single objective function in the MDAO problem, and the other involving dropping nonlinear constraints to find the correct solution within the Active manifold.

The solution is obtained by collecting snapshots of the increments. In gradient-based procedures, there is currently no equivalent for cases like ego where gradient-based optimization is not used, but efforts are being made to develop one. By analyzing snapshots of parameter corrections during iterative design, the green region can be identified by fitting it into a nonlinear approximation based on discovered features, as shown in the next slide. When dealing with multiple objective functions and solving a multiobjective MDAO problem using an ε constraint method, one instance from the Pareto front can be selected as a sacrificial lamb.

You can now optimize along one of these trajectories by removing expensive constraints to quickly solve and build your Active manifold. To build it, gather all snapshots that correct the initial values of the design parameters.

When you give data to an autoencoder, it will detect all the features contained within, focusing on the essential information. This process is similar to a nonlinear version of a singular value decomposition. The autoencoder provides a mapping that allows for the transition between a lower-dimensional parameter space and a higher-dimensional one.

For a quick example, let's move on to a multiobjective scenario.

The design analysis and optimization of your rigid CRM configuration focuses on maximizing the lift over drag ratio while ensuring that lift is as high as possible within certain constraints. This involves working in a parameter space of dimension 54, which presents challenges due to the curse of dimensionality. The CRM model includes an inviscid CFD mesh with nearly 4 million unknowns and 54 aerodynamic shape parameters that control various aspects of the aircraft's design.

When performing the MDAO, you will need to solve 17 optimization problems in order to obtain the Pareto front using the ε constraint method. This process typically takes around 32 days using 256 cores on the Flin cluster, which was established six years ago. The Pareto front illustrates the transition from solution A to solution B.

The interpretation can be seen graphically in relation to the initial wing configuration. Similar occurrences happen on the tailplane.

When running the Active manifold technique, it reduces the dimensionality of the parameter space from 58 to 10. This means going from millions to a more manageable number. The greedy procedure is used to avoid uniform sampling, allowing for effective handling of dimension 10 instead of dimension 58. Optimization problems can be solved relatively quickly in green.

By examining one optimal design solution, it is clear that the green and red results closely match except for a small variation, which is acceptable when reducing the model. In contrast, the blue results from Data regression with neural networks are not as close to the red and would require more training to match.

Considering the cost, using the Active manifold on the high dimensional model takes about 800 hours, while using it on the hyper-reduced order model only takes 84 hours. This includes all steps such as detecting the Active manifold, building and training models, and running online simulations. The speedup factor can be one to two orders of magnitude.

In a different example involving Aeroelasticity and flutter, the speedup factor was slightly more than one order of magnitude. Viscous problems are expected to be more computationally intensive. The first viscous problem tackled using this approach was suggested by Boeing Research Technology.

The problem involves a surface topology change when added to a cell in the CRM. The goal is to determine the optimal position and orientation of the change. The parameter space has dimension 21, which is viscous. By using a RANS Spalart-Allmaras Model and a wall function, we were able to reduce the dimension from 21 to four.

The main four parameters affecting the nacelle and the parameters for the shape design are shown here. The correct answer for the optimization is obtained. By focusing here, you can see how it tilts the nacelle and moves it to the optimal position.

In conclusion, Physics-informed Regression (PR) is a data-driven machine learning approach that incorporates physics principles. It is more versatile and resilient than traditional data-driven regression models, as it can handle conditions outside of the training data. PR is also interpretable, making it valuable for engineers. However, one drawback is its complexity compared to other machine learning tools.

Training costs are impacted by the curse of dimensionality, especially in higher-dimensional parameter spaces. Adaptive training methods, such as the greedy procedure, can help alleviate this issue but may not fully mitigate it. Active manifold learning, which identifies problem structures and accelerates solutions, can further improve performance. However, it currently only supports local optimization.

Despite its limitations, PR shows promise for solving complex multidisciplinary optimization problems.

PR is a data-driven physics-based machine learning method that is more versatile than data-driven regression. It is resilient to data dependency and overfitting due to the physics it incorporates, making it perform better in conditions outside of the training bounds. It is also interpretable because it learns from both models and data. However, it may seem complex as publicly available software for model reduction is not readily accessible like other machine learning tools.

In research, popularity is not a concern. Training costs are affected by the curse of dimensionality, especially in higher-dimensional parameter spaces. Adaptive training, such as the greedy procedure, can help alleviate this issue but may not fully mitigate it. An Active manifold can be used to learn the mathematical structure of the problem and accelerate the solution of MDAO problems by identifying appropriate regions for problem-solving. This approach has shown significant acceleration in solving realistic MDAO problems.

One downside of the Active manifold approach is that it currently only works with local optimization, which may require additional effort to reformulate in certain contexts.