For centuries, uncovering the mathematical laws that govern physical systems required human genius—observing a phenomenon, formulating a hypothesis, and manually deriving differential equations from first principles. However, as noted by Stefano Lenci et al., many modern complex, high-dimensional, or stochastic systems lack known governing equations. Researchers are often left with vast amounts of observational sensor data but no mathematical "rulebook" to explain the underlying dynamics.
While standard deep learning can fit these datasets to make accurate short-term predictions, it creates uninterpretable black boxes that fail to generalize outside their training distributions. The emerging field of automated governing equation discovery solves this by turning the inverse problem of physics into an optimization challenge: directly extracting human-readable, symbolic differential equations from raw data.
The Parsimony Principle and Sparse Regression
How do we find a specific equation out of an infinite space of possible mathematical expressions? The key intuition is parsimony: the physical laws governing our universe are typically simple, containing only a few interacting terms. As popularized by the SINDy framework and detailed by Alan A. Kaptanoglu et al., this problem can be framed as a sparse regression task over a dictionary of candidate functions.
Instead of guessing the exact equation, we construct a massive "library" of potential terms—such as polynomial states ($u, u^2$), spatial derivatives ($u_x, u_{xx}$), and trigonometric functions ($\sin(u)$). By evaluating these terms on our observational data, we set up a linear system:
$$\dot{u} = \Theta(U)\Xi$$
where $\Theta(U)$ is the library of candidate functions and $\Xi$ is a matrix of sparse coefficients. Because nature is parsimonious, most entries in $\Xi$ should be zero. By applying sparse optimization techniques like Lasso or sequential thresholding, the algorithm prunes away irrelevant terms, leaving only the minimal set that accurately describes the time derivative $\dot{u}$.
To handle sparse or heavily corrupted data, Yuhuang Meng demonstrates that integrating Gaussian Process Regression allows uncertainty quantification to weight the regression problem, making the discovery process significantly more robust against noise. Similarly, Sung Ha Kang et al. utilize numerical partial differential equation (PDE) schemes combined with sparse regression to identify complex equations even when the underlying coefficients change over time.
Symbolic Search and Open-Form Discovery
When the library of terms cannot be easily pre-defined, researchers turn to symbolic regression (SR) to search the space of mathematical expression trees. Instead of selecting from a fixed matrix, SR algorithms build equations from the ground up using basic mathematical building blocks (addition, multiplication, derivatives).
For example, N. Makke demonstrates the power of symbolic regression in high-energy physics by autonomously deriving analytical expressions for hadron production that closely match the complex Tsallis distribution without relying on pre-defined phenomenological structures.
To discover open-form PDEs without substantial pre-existing knowledge, Mengge Du et al. combine symbolic mathematics with reinforcement learning. Their framework generates and iteratively refines candidate equations based on how well they fit the data, embedding the discovered physics directly back into neural networks to constrain and stabilize further training.
Bridging Continuous Math and Neural Networks
What if we want to discover equations without explicitly pre-calculating derivatives from noisy data? Instead of treating neural networks as arbitrary function approximators, we can embed numerical analysis directly into their architecture.
In PDE-Net: Learning PDEs from Data, Yiping Lu et al. introduce the idea of representing spatial differential operators as constrained convolution kernels. Because a finite difference approximation of a derivative (like $\frac{\partial u}{\partial x} \approx \frac{u_{i+1} - u_i}{\Delta x}$) operates linearly on neighboring points, it looks exactly like a convolution operation. By constraining the weights of convolutional layers to match these mathematical properties, the network can learn underlying spatial derivatives through standard backpropagation.
Raw Data ---> [Constrained Convolutions] ---> Learned Derivatives (u_x, u_xx)
|
v
[Symbolic / Sparse Layer] <------------+
|
v
Discovered PDE Formula
In PDE-Net 2.0, Zichao Long et al. evolve this into a numeric-symbolic hybrid network, combining these convolutional derivative layers with a symbolic network that explicitly uncovers the algebraic formulas linking them. Taking this a step further, Adeel Pervez et al. embed a fully differentiable, sparse numerical solver directly into the learning loop. This allows the system to evaluate candidate equations by how well they simulate the physical system over time, rather than just how well they fit static snapshots.
Enforcing Symmetries and Conservation Laws
Searching for equations completely blindly is computationally inefficient and prone to overfitting. We can drastically narrow the search space by enforcing fundamental physical constraints—ensuring that the discovered equations do not violate the core rules of the universe.
One powerful constraint is the preservation of conserved quantities. Muhammad Firmansyah Kasim et al. focus on automatically identifying "constants of motion"—like energy, momentum, or angular momentum—directly from high-dimensional data without manual calculus. If we know a system must conserve energy, we can restrict our model to obey Hamiltonian mechanics. Magnus Ross et al. implement this by combining Gaussian Processes with Hamiltonian structural constraints, ensuring that every predicted trajectory strictly conserves energy while maintaining rigorous uncertainty bounds.
Furthermore, physical laws are often invariant under certain transformations, such as rotation or translation. Lexiang Hu et al. demonstrate that explicitly discovering these nonlinear symmetries from dynamic data provides the foundational stepping stones needed to reconstruct the exact mathematical form of unknown dynamics, effectively acting as a filter that discards physically impossible equations.
Mitigating Noise via Time Evolution
A major bottleneck in equation discovery is that real-world data is noisy and discrete. Calculating derivatives directly from noisy snapshots amplifies the noise dramatically, causing sparse regression to fail.
To circumvent this, modern frameworks optimize for time evolution error rather than instantaneous derivative matching. Instead of asking "does this equation match the derivative at time $t$?", they ask "if we integrate this equation forward in time, does it predict the state at time $t + \Delta t$?".
As analyzed by Haizhao Yang et al., bridging deep learning with classical linear multistep methods allows the system to infer underlying continuous forces by observing how the system evolves across multiple discrete temporal snapshots. For highly challenging scenarios, Kalpesh Sanjay More et al. introduce a Bayesian framework utilizing a spike-and-slab prior, achieving exceptional noise robustness by treating equation discovery as a probabilistic inference problem.
This strategy of combining data cleaning with advanced numerical integration is even capable of uncovering non-local phenomena. For example, Yu Xiangnan et al. successfully extract fractional differential equations (FDEs) to model anomalous particle motion and soil creep, proving that data-driven discovery can transcend standard calculus to find entirely new mathematical descriptions of complex matter.
Key Foundational Work
BACON.5: The Discovery of Conservation Laws (1981) — contributed 1 concept elements to this direction.
Discovering Admissible Simultaneous Equations of Large Scale Systems (1998) — contributed 1 concept elements to this direction.