Physics-Informed Probabilistic Learning of Linear Embeddings of Nonlinear Dynamics with Guaranteed Stability

Shaowu Pan, Karthik DuraisamySIAM J. Appl. Dyn. Syst 2020 · OpenReview

TL;DR

This paper introduces a novel framework for learning linear representations of complex, nonlinear dynamical systems using deep neural networks. It addresses the challenge of ensuring stability in these learned models and quantifying prediction uncertainties, offering a more robust and interpretable approach to analyzing and predicting the behavior of dynamic systems.

What Problem Does This Paper Address?

Nonlinear dynamical systems are ubiquitous in science and engineering, but their inherent complexity makes them challenging to analyze and predict. While the Koopman operator offers a powerful way to linearize these dynamics, learning it from data, especially for continuous-time systems, presents several hurdles. These include ensuring the stability of the learned linear representation, handling noisy data, and quantifying the uncertainty in predictions. Existing data-driven methods often struggle with these aspects, leading to unreliable long-term predictions or an inability to capture the full picture of system behavior. This paper aims to bridge this gap by developing a probabilistic, physics-informed deep learning framework that guarantees stability and provides uncertainty quantification.

Methods

The core of this work lies in a probabilistic deep learning framework designed to learn the continuous-time Koopman operator. The authors propose two formulations: a "differential form" for systems where governing equations are known, and a "recurrent form" for systems where only trajectory data is available.

Key methodological contributions include:

  • Physics-Informed Learning: The framework incorporates known physics through a "Koopman error functional" that penalizes deviations from the expected linear evolution of observables.
  • Guaranteed Stability: A novel parameterization for the Koopman operator is introduced, ensuring that its eigenvalues have non-positive real parts, thus guaranteeing the stability of the learned linear dynamics. This is achieved by leveraging the non-uniqueness of observation functions to enforce stability.
  • SVD-DMD Embedding: The authors embed Singular Value Decomposition (SVD) and Dynamic Mode Decomposition (DMD) into their neural network architecture. This helps in learning the residual dynamics with respect to dominant POD modes, improving robustness and reducing the effect of noise.
  • Mean-Field Variational Inference (MFVI): To quantify uncertainties, the paper employs MFVI. This Bayesian approach allows for learning a distribution over the model parameters (weights, biases, and Koopman operator components), enabling probabilistic predictions.
  • Automatic Differentiation Variational Inference (ADVI): The implementation leverages ADVI for efficient gradient computation and inference.

Datasets

The paper evaluates its framework on three benchmark dynamical systems:

  • A 2D fixed point attractor: A simple nonlinear system with known governing equations.
  • A 2D unforced Duffing oscillator: A classic nonlinear system.
  • Flow wake behind a cylinder: A complex, high-dimensional fluid dynamics problem, simulating a transient regime at a Reynolds number of 100. Data for this case was generated using OpenFOAM.

The authors do not introduce new datasets but rather use these established systems to demonstrate the efficacy of their approach.

Key Insights and Ideas

  • Stability is Crucial for Long-Term Prediction: The paper highlights that while unstable Koopman modes might be acceptable for short-term predictions, they lead to problematic long-term forecasts. Their guaranteed stability mechanism is a significant step towards reliable analysis.
  • SVD-DMD as a Robust Foundation: Embedding SVD-DMD provides a strong baseline for learning dynamics, allowing the neural network to focus on learning the "residual" dynamics, which can be more challenging and noisy.
  • Uncertainty Quantification is Key for Real-World Applications: By adopting a Bayesian approach, the framework provides not just point predictions but also a measure of confidence, which is essential for decision-making in complex systems.
  • Physics-Informed Learning Enhances Robustness: Incorporating physics constraints (like the Koopman error functional) helps the model generalize better, especially in the presence of noise or limited data.

Key Results

  • The proposed framework successfully learns stable Koopman decompositions for various nonlinear systems, including a complex fluid dynamics problem.
  • The method demonstrates robustness to noisy data, with predictions showing only a moderate increase in uncertainty even with significant noise levels.
  • The uncertainty quantification is shown to be effective, with predicted trajectories becoming more confident as more data is collected in relevant regions of the state space.
  • The learned Koopman eigenvalues for the Duffing oscillator and cylinder wake flow are consistent with known physical phenomena (e.g., vortex shedding frequency).

Limitations and Future Work

  • Non-convex Optimization: The optimization problem for learning the Koopman operator is non-convex, meaning the framework might converge to local minima. While they aim for global minima, this remains a challenge.
  • Computational Cost: Bayesian inference and deep learning can be computationally intensive, especially for high-dimensional systems.
  • Continuous Spectra: The current work primarily focuses on discrete spectra. The authors mention that addressing continuous spectra might require auxiliary sub-networks to characterize state-dependent eigenvalues.
  • Ideal Observation Functions: The paper notes that ideal observation functions are not uniquely determined, and neural networks might not always find the "true" set. However, their stability enforcement mechanism mitigates some of these issues.

Future work could explore more advanced variational inference techniques, methods for handling continuous spectra, and further applications to even more complex and high-dimensional systems.

Summary updated

BibTeX
@article{pan2020physics,
  author = {Shaowu Pan and Karthik Duraisamy},
  title = {{Physics-Informed Probabilistic Learning of Linear Embeddings of Nonlinear Dynamics with Guaranteed Stability}},
  journal = {SIAM J. Appl. Dyn. Syst.},
  year = {2020},
  url = {https://openreview.net/forum?id=YbgWyKYPg5},
}
Figure