Abstract
Complex dynamical systems, such as those found in climate science and biology, often rely on semi-empirical closure models to approximate unresolved small-scale processes. These models frequently suffer from structural deficiencies. This paper proposes a framework to "internalize" these errors by embedding machine learning components directly into the physical equations. Using Ensemble Kalman Inversion (EKI) for calibration, the authors demonstrate that internal error modeling produces hybrid systems that are more physically consistent and generalizable than traditional external corrections.
Introduction
In fields like fluid dynamics and atmospheric science, simulating every scale of a process is computationally prohibitive. Researchers instead use "closure models"—mathematical shortcuts representing the aggregate effects of small-scale phenomena. However, these shortcuts are often structurally flawed, assuming local or deterministic behaviors where non-local or stochastic ones exist. Traditional machine learning (ML) often attempts to replace these models entirely, but such "black-box" approaches frequently ignore fundamental physical laws like energy conservation and require vast amounts of data.
Methodology: Internal vs. External Error Modeling
The core contribution of this work is the distinction between two ways of correcting physical models with data:
- External Error Modeling: The physical model is treated as a fixed operator, and an ML-based correction term is added to the final output.
- Internal Error Modeling: The error model $\delta$ is embedded directly into the governing equations (e.g., $\dot{X} = f(X, \delta)$). This allows the correction to interact dynamically with the physical state.
To calibrate these internal models, the researchers employ Ensemble Kalman Inversion (EKI). EKI is a derivative-free optimization method, making it ideal for complex simulations where gradients are unavailable or noisy. The framework supports various error model architectures, including neural networks, dictionary learning, and stochastic extensions. A "do no harm" principle is enforced through sparsity constraints, ensuring the ML component only intervenes where the physical model is demonstrably deficient.
Experimental Validation
The framework was tested on three benchmark systems:
- Multiscale Lorenz-96: A classic model for atmospheric dynamics. The framework successfully recovered missing dynamics even when scales were not clearly separated, provided that non-local or stochastic error models were used.
- Single-scale Lorenz-96: Used to demonstrate that explicitly enforcing energy conservation within the EKI framework improves the long-term statistical accuracy (invariant measure) of the model.
- Ultradian Glucose-Insulin Model: A biological system where the framework correctly identified a missing linear term in the insulin equation while ignoring irrelevant inputs, showcasing its effectiveness even with partial observability.
Key Insights
The study highlights that internal models are fundamentally superior because they respect the underlying system dynamics. While an external correction might improve short-term predictions of observable variables, it often fails to generalize or maintain physical consistency. Furthermore, the use of sparsity-promoting regularization acts as a "shield," preventing the ML components from overfitting and ensuring they only correct genuine structural flaws.
Conclusion
By fusing domain knowledge with the flexibility of machine learning, this framework provides a robust path toward more accurate and interpretable models of complex systems. Future work will focus on scaling these methods to high-dimensional climate and fluid models, where the challenges of stability and computational cost remain significant.
Source: Learning About Structural Errors in Models of Complex Dynamical Systems (Wu et al., 2024). arXiv:2401.00035v2.
Summary updated