Abstract
Discovering governing equations from data is critical for diverse scientific disciplines as they can provide insights into the underlying phenomenon of dynamic systems. This work presents a new representation for governing equations by designing the Mathematical Operation Network (MathONet) with a deep neural network-like hierarchical structure. Specifically, the MathONet is stacked by several layers of unary operations (e.g., sin, cos, log) and binary operations (e.g., +,-), respectively. An initialized MathONet is typically regarded as a super-graph with a redundant structure, a sub-graph of which can yield the governing equation. We develop a sparse group Bayesian learning algorithm to extract the sub-graph by employing structurally constructed priors over the redundant mathematical operations. By demonstrating the chaotic Lorenz system, Lotka-Volterra system, and Kolmogorov-Petrovsky-Piskunov system, the proposed method can discover the ordinary differential equations (ODEs) and partial differential equations (PDEs) from the observations given limited mathematical operations, without any prior knowledge on possible expressions of the ODEs and PDEs.
Introduction
In science and engineering, identifying the "governing equations"—the mathematical laws describing how a system evolves—is essential for prediction and control. Traditionally, these are derived from first principles, but many complex systems lack clear underlying laws.
Existing data-driven methods face significant limitations. Symbolic regression is computationally expensive and prone to overfitting, while sparse regression techniques like SINDy require a predefined "dictionary" of candidate functions. If the true term is missing from the dictionary, the model fails. Conversely, standard neural networks act as black boxes, offering high predictive power but no interpretability. The researchers behind MathONet aim to bridge this gap by creating a system that learns the structure of mathematical operations directly from data.
Methodology: The MathONet Framework
The authors introduce a novel architecture called the Mathematical Operation Network (MathONet). Instead of standard neurons, MathONet utilizes layers of unary operations (e.g., $\sin, \cos, \log, \exp$) and binary operations (e.g., $+, -, \times$).
1. Network Architecture
MathONet is structured similarly to a DenseNet, where the output of every layer is "pasted" into all subsequent layers. This allows the network to build increasingly complex compositions (such as $\sin(x^2 + y)$) from basic building blocks. The network begins as an over-parameterized "super-graph"—a dense web of all possible mathematical combinations.
2. Sparse Group Bayesian Learning
To identify the true governing equation, the redundant connections in the super-graph must be pruned. The authors developed a Bayesian approach using Laplace approximation to assign "sparsity priors" to weights. As the model trains, it calculates the "evidence" for each operation. If an operation does not significantly help explain the data, its weight is driven to zero, and it is pruned from the graph. This frames scientific discovery as a network compression problem.
Key Results and Insights
The model was evaluated on classic benchmarks, including the chaotic Lorenz system, the Lotka-Volterra predator-prey model, and the Fisher-KPP partial differential equation (PDE).
Discovery as Compression
In the Lorenz system experiment, the model started with 1,551 possible terms. After six cycles of Bayesian pruning, it successfully reduced the model to the exact two terms required for the $z$-axis equation, significantly outperforming standard techniques like Sparse Group Lasso.
Robustness and PDE Discovery
The method remained accurate even with significant Gaussian noise added to the data, identifying correct terms within 0.5% of their true values. Furthermore, by combining MathONet with CNN-style spatial stencils, the model successfully discovered the Fisher-KPP equation, identifying both the diffusion coefficient and the growth polynomial accurately.
Conclusion
MathONet represents a shift from picking functions from a list to composing them from fundamental operations. While the current model requires users to define initial network depth and faces computational costs related to Hessian matrix calculations, it provides a principled, Bayesian way to discover interpretable laws of nature from raw observations.
Summary updated