In the standard machine learning paradigm, we often settle for "black boxes"—models like deep neural networks that make accurate predictions but hide their logic within millions of weights. However, in the natural sciences, the goal is rarely just prediction; it is understanding. We seek the concise, symbolic laws—like $F=ma$ or $E=mc^2$—that govern the universe.
Symbolic Regression (SR) is the research direction aimed at bridging this gap. As reviewed by Nour Makke, SR treats the discovery of physical laws as a search problem over the space of mathematical expressions. Instead of fitting parameters to a fixed curve, these algorithms "assemble" equations from a library of primitives (like $+$, $\times$, $\sin$, or $\exp$) to find the simplest formula that explains the observed data.
The Library Bottleneck
The central mechanism in most equation discovery tools, such as the SINDy framework or the work by Samuel Rudy et al., is sparse regression. The algorithm is given a "library" of candidate terms—for example, $x, x^2, \sin(x), \dot{x}$. It then tries to find the smallest subset of these terms that can reconstruct the system's dynamics.
However, a recurring theme across nearly 160 papers is the Library Bottleneck. As noted by Niklas Heim et al. and Dongxiao Zhang and Hao Xu, if the true governing law involves a mathematical structure not present in the predefined library—such as a fractional derivative or a specific transcendental function—the discovery process fails. It either produces a messy, non-physical approximation or fails to generalize entirely. This constraint has driven the field toward more flexible, "generative" approaches to equation building.
From Genetic Algorithms to Neural Guidance
Historically, symbolic regression relied on Genetic Programming, where equations "evolve" through mutation and crossover. While powerful, this search is often inefficient and prone to "bloat" (producing overly complex equations).
Modern approaches have introduced a "deep learning revolution" to guide this search:
- Recursive Simplification: Max Tegmark and Silviu-Marian Udrescu developed AI Feynman, which uses neural networks to "test" data for symmetries and separability. If a neural network can learn that a dataset is periodic, the symbolic search can focus exclusively on trigonometric functions, drastically narrowing the search space.
- Strategic Search: Frameworks like the Symbolic Physics Learner (SPL) by Fangzheng Sun et al. and RSRM by Yilong Xu and Hao Sun treat equation discovery like a game of chess. They use Monte Carlo Tree Search (MCTS) to navigate the tree of possible mathematical expressions, using reinforcement learning to "sense" which branches are likely to lead to a physically sound law.
- Transformer-based Generation: Newer models, such as those proposed by Haohan Zou et al., use transformers to "translate" numeric trajectories directly into symbolic strings, treating the discovery of a Lyapunov function or an ODE as a sequence-to-sequence task.
Discovering Physics from Raw Perception
One of the most ambitious frontiers in this field is moving from structured data (like tables of positions) to raw sensory input. Lele Luan et al. and Siyu Huang et al. have demonstrated pipelines that can "watch" a video of a swinging pendulum or a falling ball and derive the underlying ODEs.
These systems typically use a Variational Auto-Encoder (VAE) to compress the video into a low-dimensional "latent space" that represents the physical state (like angle and velocity). A symbolic regression module then operates on this latent space to find the governing equations. This allows for the discovery of laws in systems where we don't even know which variables are important yet.
Why It Works: The Principle of Parsimony
The reason these methods can recover true physical laws from noisy data is the Minimum Description Length (MDL) principle, a concept championed early on by R. Bharat Rao and Stephen C. Y. Lu.
Nature is generally "sparse." While a high-degree polynomial can fit any curve perfectly, it is rarely the true physical explanation. By penalizing complexity (the "length" of the equation) and rewarding accuracy, symbolic regression algorithms exploit the fact that the true law is usually the simplest one that doesn't overfit. This structural prior is what allows symbolic models to achieve out-of-domain generalization. As argued by Niclas Alexander Göring et al., while a black-box neural network might fail the moment a system enters an unobserved state, a correctly discovered symbolic law like $1/r^2$ remains valid across the entire universe.