Finite Expression Methods for Discovering Physical Laws from Data

Zhongyi Jiang, Chunmei Wang, Haizhao Yang

Abstract

Nonlinear dynamics is a pervasive phenomenon observed in scientific and engineering disciplines. However, the task of deriving analytical expressions to describe nonlinear dynamics from limited data remains challenging. In this paper, we shall present a novel deep symbolic learning method called the "finite expression method" (FEX) to discover governing equations within a function space containing a finite set of analytic expressions, based on observed dynamic data. The key concept is to employ FEX to generate analytical expressions of the governing equations by learning the derivatives of partial differential equation (PDE) solutions through convolutions. Our numerical results demonstrate that our FEX surpasses other existing methods (such as PDE-Net, SINDy, GP, and SPL) in terms of numerical performance across a range of problems, including time-dependent PDE problems and nonlinear dynamical systems with time-varying coefficients. Moreover, the results highlight FEX's flexibility and expressive power in accurately approximating symbolic governing equations.

Introduction

For centuries, physicists derived the laws of nature through intuition and rigorous mathematical derivation. Today, we are drowning in data from complex systems—climate patterns, neural activity, financial markets—where the underlying "rules" remain hidden. While deep learning can predict these systems, it often acts as a "black box," providing answers without explanations.

The field of Symbolic Regression aims to find human-readable mathematical expressions that fit a dataset. However, current methods face significant challenges. Sparse regression (e.g., SINDy) requires a pre-defined dictionary; Genetic Programming (GP) struggles with fine-tuning constants; and standard Neural Networks lack interpretability. The Finite Expression Method (FEX) addresses these limitations by discovering governing equations within a function space of finite analytic expressions.

Comparison of FEX with existing symbolic modeling methods like PDE-Net, SINDy, GP, and SPL, highlighting its advantages in handling varying coefficients and accuracy.

Methodology

The core innovation of FEX is representing mathematical equations as binary expression trees. This framework combines Reinforcement Learning (RL) for structural discovery with continuous optimization for parameter refinement.

The FEX framework: (a) The searching loop integrating data, loss, and RL-based updates; (b) The expression generation process using a controller to sample operators for the tree.

The FEX Framework

  1. Tree Structure: Each node in the tree is either an operator (e.g., $+$, $\times$, $\sin$) or an operand (e.g., a variable or a derivative).
  2. Learning Derivatives via Convolutions: Instead of selecting derivatives from a discrete list, FEX uses trainable convolution kernels. This "partially relaxed" approach treats derivative discovery as a continuous optimization problem, allowing gradient-based learning.
  3. RL Controller: A Reinforcement Learning agent acts as the architect, proposing different tree structures based on a score derived from the loss.
  4. Hybrid Optimization: Once a structure is proposed, the model employs high-performance optimizers like Adam and BFGS to fine-tune constants and convolution kernels.

Experimental Results

FEX was evaluated against four challenging benchmarks: the 2D Burgers’ Equation (with both constant and time-varying coefficients), the 2D Hopf Normal Form, and the JMAK Nonlinear Equation.

Performance Comparison

FEX consistently outperformed established methods:

  • Accuracy: In the 2D Hopf normal form test, FEX achieved a Mean Absolute Error (MAE) of $\sim 5.5 \times 10^{-3}$, significantly lower than PDE-Net. SINDy and GP failed to recover the correct structure entirely in complex cases.
  • Robustness: FEX maintained high accuracy even with 1% noise in the training data, whereas other methods showed significant error spikes.
  • Generalization: FEX was the only method capable of correctly identifying the equation structure for all test cases, including the complex JMAK equation and varying-coefficient Burgers' equation.

Discussion and Conclusion

A key insight from FEX is the transition from purely discrete to continuous search for derivatives. By using convolution filters, the authors transformed a difficult search problem into a smooth landscape navigable by gradient descent. Furthermore, FEX's reliance on a small set of basic operators makes it highly memory-efficient, building complexity through tree depth rather than massive dictionaries.

While FEX is powerful, it faces challenges with computational costs as tree depth increases and with extremely high-order derivatives in high-dimensional spaces. Future work may extend FEX to stochastic differential equations or integrate it into real-time physics-informed pipelines for noisy experimental data. In summary, FEX represents a significant step forward in "rediscovering" physics from raw observations, providing both accuracy and interpretability.

Summary updated

Abstract

Nonlinear dynamics is a pervasive phenomenon observed in scientific and engineering disciplines. However, the task of deriving analytical expressions to describe nonlinear dynamics from limited data remains challenging. In this paper, we shall present a novel deep symbolic learning method called the "finite expression method" (FEX) to discover governing equations within a function space containing a finite set of analytic expressions, based on observed dynamic data. The key concept is to employ FEX to generate analytical expressions of the governing equations by learning the derivatives of partial differential equation (PDE) solutions through convolutions. Our numerical results demonstrate that our FEX surpasses other existing methods (such as PDE-Net, SINDy, GP, and SPL) in terms of numerical performance across a range of problems, including time-dependent PDE problems and nonlinear dynamical systems with time-varying coefficients. Moreover, the results highlight FEX's flexibility and expressive power in accurately approximating symbolic governing equations.
BibTeX
@misc{jiang2023finite,
  author = {Zhongyi Jiang and Chunmei Wang and Haizhao Yang},
  title = {{Finite Expression Methods for Discovering Physical Laws from Data}},
  year = {2023},
  eprint = {2305.08342},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2305.08342},
}
Figure