TL;DR
This paper introduces a novel data-driven framework for discovering "closure models" – essential components for simplifying complex dynamical systems. By leveraging sparse polynomial regression and artificial neural networks, the authors can automatically learn these missing pieces from data, enabling accurate predictions for systems that were previously intractable.
What Problem Does This Paper Address?
Many real-world systems, from fluid dynamics to molecular simulations, are governed by incredibly complex, high-dimensional equations. To make these systems computationally tractable, researchers often resort to "reduced order models" (ROMs). However, these simplified models inevitably lose information, and the impact of this lost information needs to be accounted for through a "closure model." Traditionally, these closure models are derived using physical intuition or complex mathematical arguments, which can be difficult, time-consuming, and sometimes impossible for novel systems. This paper addresses the challenge of automatically discovering these closure models directly from observational data, without relying on pre-existing physical assumptions about their form.
Methods
The core of the paper is the "operator inference" framework. This approach augments the reduced-order state with a new set of variables representing the closure term. The evolution of both the reduced state and the closure term is then learned from data. Two primary methods are explored for learning the closure operator:
- Sparse Polynomial Regression (SINDy-like): This method assumes the closure can be represented as a sparse combination of polynomial terms. It uses lasso regression to identify the most important polynomial features and their coefficients, effectively performing feature selection and parameter estimation simultaneously.
- Artificial Neural Networks (ANNs): This approach uses feedforward neural networks with multiple hidden layers to learn the complex, potentially non-linear relationship of the closure term. The network is trained to predict the future state of the closure based on past states of both the reduced system and the closure itself.
A key innovation is the ability of this framework to capture "non-Markovian" (memory) effects, meaning the closure's future behavior depends not just on the current state but also on its past history.
Datasets
The authors evaluate their framework on several benchmark dynamical systems:
- A 3D linear system.
- The Van der Pol (VdP) oscillator.
- The Duffing map (a chaotic system).
- The Lorenz system (both chaotic and non-chaotic cases).
- A 1D viscous Burgers equation (a partial differential equation).
For each system, data is generated from the full-order model, with a portion used for training and the remainder for testing. The specific number of snapshots and simulation times vary per system, as detailed in the results sections.
Key Insights and Ideas
- Data-Driven Closure is Feasible: The paper demonstrates that complex closure models, which are often derived through laborious physical reasoning, can be effectively learned directly from data.
- Capturing Memory Effects: The framework naturally incorporates "memory" or non-Markovian effects, which are crucial for accurate closure modeling in many systems. This is a significant advantage over simpler ROMs.
- Polynomials vs. ANNs: Both sparse polynomial regression and ANNs prove effective, with ANNs showing more robustness for highly complex or chaotic systems where polynomial approximations might struggle.
- Compact Memory for Certain Systems: Theoretical results suggest that for specific classes of nonlinear systems, the memory required for closure can be surprisingly compact, meaning only a limited history of past states is needed.
- Predictive Power: The primary goal is prediction on unseen data, and the results show that these data-driven closures enable accurate forecasting, even for chaotic systems.
Key Results
The framework was successfully applied to a range of systems, demonstrating its versatility:
- Linear System: The polynomial closure model accurately reproduced the dynamics, with the lasso path identifying a sparse set of relevant terms.
- Van der Pol System: Both polynomial regression and ANNs achieved excellent a posteriori performance, matching the true dynamics closely.
- Duffing Map (Chaotic): The model demonstrated excellent correspondence in Lyapunov exponents and correlation dimensions, and a Diks test confirmed the attractor reconstruction was close to the ground truth.
- Lorenz System: The ANN model performed well for both chaotic and non-chaotic cases, with metrics aligning with the truth.
- Burgers Equation: The ANN model showed superior performance on unseen data compared to polynomial closures, highlighting its ability to handle extrapolation better.
Limitations and Future Work
- Extrapolation Challenges for Polynomials: The paper notes that high-order polynomial features can lead to extrapolation issues when predicting unseen data, particularly for chaotic systems. ANNs, being more akin to infinite polynomial series, tend to be more robust in this regard.
- Hyperparameter Tuning: Both polynomial regression (e.g., penalty coefficient $\lambda$) and ANNs (e.g., number of layers, units, activation functions) require careful hyperparameter selection, which can be computationally intensive.
- Computational Cost: While the framework aims to reduce complexity, training ANNs for high-dimensional systems can still be computationally demanding.
- Future Work: The authors suggest exploring implicit-SINDy for non-rational functions and further investigating the theoretical underpinnings of compact memory in closure dynamics. The provided code repository indicates ongoing development and potential for broader application.
Summary updated