Deep learning has historically faced a "black box" problem in the natural sciences. While a neural network can predict the trajectory of a pendulum or the spread of a fluid with high accuracy, it typically does so through millions of opaque parameters that offer no insight into the underlying physics.
A growing body of research is shifting this paradigm by moving from pure interpolation to symbolic discovery. The goal is to design architectures that don't just mimic data, but distill it into the human-readable language of mathematics—Ordinary Differential Equations (ODEs), Partial Differential Equations (PDEs), and conservation laws.
Constraining the Architecture to Force Interpretability
The most direct way to make a model interpretable is to restrict its "vocabulary" to physical operations. Instead of using generic layers, researchers are building networks where the internal components correspond to mathematical operators.
In PDE-Net: Learning PDEs from Data, the authors represent differential operators as constrained convolution kernels. By forcing the network to learn these specific kernels, the model is effectively "writing" a PDE as it trains. Similarly, Suresh Bishnoi and colleagues developed Hamiltonian Graph Neural Networks, which bake the laws of Hamiltonian mechanics directly into the graph structure. Because the model is required to conserve energy by design, the learned representations can be directly translated into physical potentials, such as the Lennard-Jones potential used in molecular dynamics.
This "physics-by-design" approach extends to specialized domains. Heming Zhang developed consDeepSignaling, which constrains deep learning architectures using known biological signaling pathways. By mirroring the structure of a cell's internal logic, the model's predictions about drug responses become biologically verifiable rather than statistically mysterious.
From Neural Weights to Symbolic Formulas
If a standard neural network is a "black box," symbolic regression is an attempt to turn it into a "glass box." The objective is to find a concise mathematical expression—like $F = ma$—that fits the data.
Recent breakthroughs have scaled this process. Owen Dugan and Rumen Dangovski introduced OccamNet, which uses a reinforcement learning-based sampling strategy to find compact formulas. Rather than optimizing weights, the network optimizes the probability of selecting different mathematical symbols.
For dynamical systems, Juliane Weilbach and her team developed DynAIFeynman, which can observe a system's trajectory and automatically derive the governing ODEs. This allows researchers to move from "watching" a video of a system to "reading" its laws. Lele Luan even demonstrated that this can be done directly from raw video pixels, mapping visual coordinates to physical ones while simultaneously identifying external forces.
Bypassing the Curse of Dimensionality
A major hurdle in scientific discovery is the "curse of dimensionality"—as systems get more complex, the data required to model them grows exponentially. Ziming Liu and James Halverson proposed Kolmogorov-Arnold Networks (KANs) as a solution. Unlike standard Multi-Layer Perceptrons that use fixed activation functions on nodes, KANs use learnable spline functions on edges. This allows them to represent complex functions with far fewer parameters, making it easier for scientists to "read out" the symbolic relationships the network has discovered.
In fluid dynamics, where systems are notoriously high-dimensional, Tianning Tang used symbolic regression to discover new boundary equations for wave breaking. The resulting human-readable PDE revealed a physical decoupling between surface and fluid velocities that had previously been obscured by the complexity of the data.
Learning the Response: Green's Functions and Operators
Beyond discovering equations, some researchers focus on discovering the fundamental response of a system. Nicolas Boullé has pioneered the use of rational neural networks to learn Green’s functions—mathematical tools that describe how a system reacts to a point source.
By training a model to learn these functions, the AI reveals the system's underlying symmetries and conservation laws. This approach was further refined by Jianghang Gu in the Green’s Operator Network (GON), which provides an explainable framework for solving 3D PDEs. Instead of just giving an answer, the GON shows how every part of the input field influences the final output, providing a causal map of the physical process.
Why It Works: The Manifold of Physical Laws
The success of these methods suggests that physical laws exist on a low-dimensional manifold within the vast space of all possible mathematical expressions. By using architectures like EQL÷ (developed by Subham S. Sahoo) or FlowMixer, researchers are essentially "shrinking" the search space of the neural network to only include physically plausible solutions.
When we force a model to be symbolic, we aren't just making it easier to understand; we are making it more robust. A symbolic equation discovered by Tapas Tripura can extrapolate to unseen scenarios where a standard "black box" would fail, because the equation captures the universal principle rather than just the local correlation. This transition from curve-fitting to law-finding marks the beginning of a new era where AI acts as a collaborator in scientific theory-building.