Machine Discovery of Partial Differential Equations from Spatiotemporal Data

Ye Yuan, Junlin Li, Liang Li, Frank Jiang, Xiuchuan Tang, Fumin Zhang, Sheng Liu, Jorge M. Gonçalves, Henning U. Voss, Xiuting Li, Jürgen Kurths, Han DingarXiv 2019 · OpenReview

TL;DR

This paper introduces Sparse Spatiotemporal System Discovery (S³d), a novel framework for automatically discovering underlying Partial Differential Equations (PDEs) from spatiotemporal data. By leveraging Sparse Bayesian Learning, S³d efficiently identifies essential physical terms, leading to parsimonious, accurate, and interpretable models, which can accelerate scientific discovery in complex fields.

What Problem Does This Paper Address?

Discovering the fundamental physical laws governing complex phenomena, often expressed as Partial Differential Equations (PDEs), is a cornerstone of scientific progress. Traditionally, this process relies heavily on human intuition, extensive theoretical work, and laborious trial-and-error experimentation. While powerful, this approach can be time-consuming and may struggle with systems exhibiting highly complex dynamics or where a priori knowledge is limited. This paper addresses the challenge of automating this discovery process, aiming to extract governing PDEs directly from observed spatiotemporal data, thereby assisting physicists and researchers in understanding intricate systems more efficiently.

Methods

The core of the proposed method is Sparse Spatiotemporal System Discovery (S³d), which builds upon Sparse Bayesian Learning (SBL). S³d formulates the PDE discovery problem as a regression task. A comprehensive library of candidate physical terms (e.g., polynomial terms, derivatives, and their combinations) is constructed to represent the unknown function in the PDE. SBL is then employed to identify the most relevant terms by enforcing sparsity, effectively selecting a parsimonious set of terms that best explain the observed data.

A key novelty lies in the optimization strategy for the SBL cost function. The authors propose a Difference of Convex (DC) programming approach solved iteratively using a Concave-Convex Procedure (CCP). This approach allows for efficient optimization in both the coefficient space and the latent variable space, leading to a more robust and accurate identification of sparse solutions. The paper also details methods for estimating derivatives from noisy data, including polynomial approximation and the Pade scheme, which are crucial for constructing the dictionary matrix.

Datasets

The paper evaluates S³d on a variety of datasets:

  • Synthetic data: Generated from eight prototypical PDEs, including:
  • Sine-Gordon equation
    • Fisher's equation
    • Korteweg-de Vries (KdV) equation
    • FitzHugh-Nagumo equation
    • Nonlinear Schrödinger equation
    • Klein-Gordon equation
    • Kuramoto-Sivashinsky (KS) equation
    • Navier-Stokes equations
  • Experimental data: Real-world data from a traveling-wave (TW) convection experiment in binary fluids, specifically focusing on the Complex Ginzburg-Landau Equation (CGLE).

The synthetic datasets were generated with varying levels of added Gaussian noise (e.g., 1%) to test the robustness of the method.

Key Insights and Ideas

  • Automated PDE Discovery: S³d automates the identification of governing PDEs from spatiotemporal data, significantly reducing the reliance on human intuition and manual experimentation.
  • Sparsity for Parsimony and Interpretability: By enforcing sparsity through Sparse Bayesian Learning, S³d discovers parsimonious models, meaning only the essential physical terms are retained. This enhances interpretability, making the discovered equations more understandable to physicists.
  • Robust Derivative Estimation: The paper highlights the critical role of accurate derivative estimation from noisy data. The proposed use of polynomial interpolation and Pade schemes proves effective in handling noise, which is a common challenge in real-world data.
  • DC Programming for SBL: The novel DC programming approach for optimizing the SBL cost function offers a more effective way to find sparse solutions compared to traditional methods.
  • Potential for Accelerating Scientific Discovery: By providing a tool to quickly identify governing equations, S³d has the potential to accelerate the discovery of new physical laws and deepen our understanding of complex phenomena across various scientific domains.

Key Results

The paper demonstrates S³d's effectiveness across numerous examples:

  • Prototypical PDEs: S³d successfully rediscovers several well-known PDEs (e.g., Navier-Stokes, sine-Gordon, KdV, Fisher's) from synthetic data, often with high accuracy, even in the presence of noise. For instance, in the Sine-Gordon equation discovery, an MSE of 0.0706% was achieved with noise-free data.
  • Experimental Data: The method successfully discovers the Complex Ginzburg-Landau Equation (CGLE) from experimental data of traveling-wave convection, providing a quantitative explanation for observed dynamics.
  • Comparison with State-of-the-Art: S³d outperforms or matches existing methods like STRidge and PDE-FIND in terms of accuracy and robustness, particularly in handling noisy data and identifying complex dynamics. For example, in a comparison with PDE-FIND on the Quantum Harmonic Oscillator and Nonlinear Schrödinger equations, PDE-FIND failed to discover the correct equations, while S³d succeeded.
  • Efficiency: S³d often requires fewer data samples for training compared to other methods, demonstrating its efficiency.

Limitations and Future Work

  • Data Requirements: While S³d can work with subsampled data, the quality and representativeness of the data are crucial for successful discovery. The paper notes that small samples including essential information are key.
  • Observability: The current method relies on estimating temporal and spatial derivatives of measured state variables. There are cases where system variables might not be directly observable, which is a limitation.
  • Parameter Linearity: The current method is primarily designed for discovering equations that are linear with respect to their parameters.
  • Future Work: The authors suggest extending the method to handle cases where system variables are not directly observable and to discover equations that are nonlinear with respect to their parameters. They also plan to explore its application to a wider range of practical problems and to further unify the discovery of natural laws.

Summary updated

Abstract

The study presents a general framework for discovering underlying Partial Differential Equations (PDEs) using measured spatiotemporal data. The method, called Sparse Spatiotemporal System Discovery ($\text{S}^3\text{d}$), decides which physical terms are necessary and which can be removed (because they are physically negligible in the sense that they do not affect the dynamics too much) from a pool of candidate functions. The method is built on the recent development of Sparse Bayesian Learning; which enforces the sparsity in the to-be-identified PDEs, and therefore can balance the model complexity and fitting error with theoretical guarantees. Without leveraging prior knowledge or assumptions in the discovery process, we use an automated approach to discover ten types of PDEs, including the famous Navier-Stokes and sine-Gordon equations, from simulation data alone. Moreover, we demonstrate our data-driven discovery process with the Complex Ginzburg-Landau Equation (CGLE) using data measured from a traveling-wave convection experiment. Our machine discovery approach presents solutions that has the potential to inspire, support and assist physicists for the establishment of physical laws from measured spatiotemporal data, especially in notorious fields that are often too complex to allow a straightforward establishment of physical law, such as biophysics, fluid dynamics, neuroscience or nonlinear optics.
BibTeX
@misc{yuan2019machine,
  author = {Ye Yuan and Junlin Li and Liang Li and Frank Jiang and Xiuchuan Tang and Fumin Zhang and Sheng Liu and Jorge M. Gonçalves and Henning U. Voss and Xiuting Li and Jürgen Kurths and Han Ding},
  title = {{Machine Discovery of Partial Differential Equations from Spatiotemporal Data}},
  year = {2019},
  eprint = {1909.06730},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1909.06730},
}
Figure