Discovering governing equations from data by sparse identification of nonlinear dynamical systems

Steven L. Brunton, Joshua L. Proctor, J. Nathan KutzProceedings of the National Academy of Sciences 2016 · DOI

TL;DR

This paper introduces a new algorithm, SINDy (Sparse Identification of Nonlinear Dynamics), that automatically discovers the governing equations of a system from data. By assuming that physical laws are sparse (i.e., composed of a few key terms), SINDy uses sparse regression to efficiently identify these terms, leading to parsimonious and accurate models. This approach has been successfully applied to a wide range of complex systems, including chaotic attractors and fluid dynamics, even uncovering dynamics that eluded experts for decades.

What Problem Does This Paper Address?

A fundamental goal in science and engineering is to understand the underlying physical laws that govern complex systems. While we are generating vast amounts of data across fields like climate science, neuroscience, and fluid dynamics, the governing equations often remain elusive. Traditional system identification methods often require strong assumptions about the model's structure or are computationally prohibitive for high-dimensional systems. This leaves a gap in our ability to extract meaningful, predictive physical models from the data we collect. The authors aim to bridge this gap by developing a method that can discover these governing equations with minimal prior assumptions, relying on the inherent sparsity of many physical laws.

Methods

The core of the paper is the SINDy algorithm, which leverages sparse regression and compressed sensing principles. The key idea is that the governing equations of a physical system, when expressed in a suitable basis of candidate functions (e.g., polynomials, trigonometric functions), are typically sparse – meaning only a few terms are non-zero.

Here's a breakdown of the approach:

  1. Data Collection: Time-series data of the system's state variables ($x(t)$) and their time derivatives ($\dot{x}(t)$) are collected.
  2. Library Construction: A library of candidate nonlinear functions of the state variables is created (e.g., $1, x, y, z, x^2, xy, \sin(x)$, etc.). This forms a matrix $\Theta(X)$.
  3. Sparse Regression: The problem is framed as finding a sparse coefficient vector $\Xi$ such that $\dot{X} \approx \Theta(X)\Xi$. This is solved using sparse regression techniques (similar to LASSO), which penalize non-zero coefficients, effectively selecting only the most important terms from the library.
  4. Model Identification: The non-zero coefficients in $\Xi$ directly reveal the terms present in the governing equations.

The authors also discuss extensions for handling noisy data (using sequential thresholded least-squares), high-dimensional systems (via dimensionality reduction like SVD), and parameterized or time-varying systems.

Datasets

The paper demonstrates the SINDy algorithm on several well-established benchmark systems and a complex real-world problem:

  • Canonical Systems:
    • Linear and nonlinear damped oscillators (linear and cubic dynamics).
  • The chaotic Lorenz system.
  • A three-dimensional linear system.
  • Complex Systems:
    • Fluid wake behind a cylinder (using direct numerical simulation data).
  • Parameterized Systems:
    • The logistic map (with stochastic forcing).
  • The Hopf normal form.

No new datasets were introduced; instead, the authors used data generated from known systems or from established simulation techniques.

Key Insights and Ideas

  • Sparsity as a Guiding Principle: The core assumption that physical laws are sparse in a suitable function space is incredibly powerful. It allows for efficient identification of governing equations without needing to know the exact form beforehand.
  • Uncovering Decades-Old Dynamics: The fluid wake example is particularly striking. SINDy was able to discover the underlying dynamics of vortex shedding behind a cylinder, a problem that took experts nearly 30 years to resolve, highlighting the method's ability to tackle complex, emergent phenomena.
  • Beyond Prediction: Capturing Attractors: For chaotic systems like the Lorenz system, accurately predicting individual trajectories is often impossible due to sensitivity to initial conditions. SINDy's success lies in its ability to capture the attractor dynamics, which is crucial for understanding the system's long-term behavior.
  • Generalizability: The method's ability to handle noisy data, high-dimensional systems (via dimensionality reduction), and parameterized/time-varying systems demonstrates its broad applicability across diverse scientific domains.

Key Results

  • Accurate Model Identification: SINDy successfully identified the correct governing equations for all tested systems, including linear and nonlinear oscillators, the Lorenz system, and the fluid wake dynamics.
  • Parsimonious Models: The algorithm consistently produced sparse models with a minimal number of terms, balancing accuracy with simplicity and avoiding overfitting.
  • Robustness to Noise: The method showed remarkable robustness to noise in the derivative measurements, accurately identifying dynamics even when velocities were approximated from noisy data. For the Lorenz system, coefficients were identified to within 0.03% of true values, and attractors were well-preserved even with significant noise.
  • Fluid Dynamics Breakthrough: In the fluid wake example, SINDy identified quadratic nonlinearities that correctly represent the dynamics on the slow manifold, a key insight that resolved a long-standing debate in the field.
  • Parameter Discovery: The method successfully identified parameterized normal forms for the logistic map and the Hopf normal form, demonstrating its utility in understanding systems with bifurcations.

Limitations and Future Work

  • Choice of Function Space: The paper notes that identifying the "correct" approximating function space is crucial for achieving sparsity. The choice of library (e.g., polynomials up to a certain order) can impact the sparsity and accuracy of the identified model. Future work could explore adaptive methods for selecting or transforming the function space to maximize sparsity.
  • Degeneracy in High-Order Polynomials: For linear systems, including high-order polynomial terms can introduce degeneracy, where linear combinations of powers of $x$ approximate other exponential rates. This suggests a need for hierarchical approaches to identification.
  • Dynamical Symmetries and Conserved Quantities: The authors mention that many systems possess dynamical symmetries and conserved quantities that could alter the form of identified dynamics. Future research could explore how to incorporate these properties into the identification process.
  • Data Quality and Quantity: While robust to noise, the performance of SINDy, like any data-driven method, will ultimately depend on the quality and quantity of the available data.

The authors envision SINDy as a significant step towards intelligent, unassisted identification of dynamical systems, applicable to a wide range of fields where data is abundant but governing equations are not.

Summary updated

Abstract

Extracting governing equations from data is a central challenge in many diverse areas of science and engineering. Data are abundant whereas models often remain elusive, as in climate science, neuroscience, ecology, finance, and epidemiology, to name only a few examples. In this work, we combine sparsity-promoting techniques and machine learning with nonlinear dynamical systems to discover governing equations from noisy measurement data. The only assumption about the structure of the model is that there are only a few important terms that govern the dynamics, so that the equations are sparse in the space of possible functions; this assumption holds for many physical systems in an appropriate basis. In particular, we use sparse regression to determine the fewest terms in the dynamic governing equations required to accurately represent the data. This results in parsimonious models that balance accuracy with model complexity to avoid overfitting. We demonstrate the algorithm on a wide range of problems, from simple canonical systems, including linear and nonlinear oscillators and the chaotic Lorenz system, to the fluid vortex shedding behind an obstacle. The fluid example illustrates the ability of this method to discover the underlying dynamics of a system that took experts in the community nearly 30 years to resolve. We also show that this method generalizes to parameterized systems and systems that are time-varying or have external forcing.
BibTeX
@inproceedings{brunton2016discovering,
  author = {Steven L. Brunton and Joshua L. Proctor and J. Nathan Kutz},
  title = {{Discovering governing equations from data by sparse identification of nonlinear dynamical systems}},
  booktitle = {Proceedings of the National Academy of Sciences},
  year = {2016},
  doi = {10.1073/pnas.1517384113},
}
Figure