When a standard neural network is trained to predict the motion of a pendulum, it treats the problem as a sequence-to-sequence task. It learns that if the pendulum is at position $A$, it will likely be at position $B$ next. However, because these "black-box" models lack an understanding of the underlying physics, they inevitably suffer from numerical energy drift. Over long simulations, the pendulum might gain energy out of nowhere, swinging higher and higher until it spins in circles, or lose energy and come to an unnatural halt.
To solve this, a growing body of research suggests we should not ask neural networks to predict trajectories. Instead, we should ask them to learn the energy manifold of the system.
The Energy-First Inductive Bias
The core intuition behind Hamiltonian and Lagrangian neural networks is to embed the laws of classical mechanics directly into the network's architecture. Instead of the network outputting a velocity or a next state, it outputs a single scalar value: the total energy of the system.
In Deep Lagrangian Networks (DeLaN), Michael Lutter and Jan Peters demonstrate that by forcing the network to represent kinetic and potential energy separately, the model can derive the equations of motion using the Euler-Lagrange equation. Because the motion is derived from these fundamental principles, the resulting predictions are guaranteed to be physically plausible.
Similarly, Hamiltonian Neural Networks (HNN) use automatic differentiation to find the derivatives of a learned Hamiltonian (total energy) with respect to position and momentum. As Sarvin Moradi and Roland Tóth explain, this ensures the model respects the geometric properties of phase space, leading to near-perfect energy conservation over thousands of timesteps.
Why It Works: Conservation as a Constraint
The reason these models generalize so well—often with 1% of the parameters of a standard model—is that they are mathematically incapable of violating conservation laws.
- Stability: In Sparse Symplectically Integrated Neural Networks, Daniel M. DiPietro and Bo Zhu show that by combining a learned Hamiltonian with a symplectic integrator, the model stays on the "energy surface" indefinitely.
- Data Efficiency: Because the network only needs to learn the scalar energy function rather than every possible state transition, it requires far less data. Yusuke Tanaka has shown that these models remain accurate even when training data is sparse or noisy.
- Interpretability: The learned latent space isn't a random vector; it represents physical quantities like mass matrices and potential fields. Christine Allen-Blanchette demonstrated in Hamiltonian GAN that this allows models to discover the "source code" of a system's physics directly from video frames.
Scaling to Complex Structures
While a single pendulum is simple, real-world systems are often articulated, like robotic arms or molecular chains. Ravinder Bhattoo and Sayan Ranu introduced Lagrangian Graph Neural Networks (LGNN) to handle this. By treating the system as a graph where nodes are masses and edges are constraints, they achieved "zero-shot" generalization—training on a 4-link chain and successfully simulating a 100-link chain.
This modularity is a recurring theme. Yupu Lu and Jia Pan proposed ModLaNets, which break complex systems into reusable physical components. This allows AI to learn the dynamics of simple parts and "assemble" them to predict the behavior of entirely new, complex machines.
Beyond Ideal Physics: Dissipation and Fields
Pure Hamiltonian systems are "conservative"—they assume no friction. To bridge the gap to the messy real world, researchers like Andrew Sosanya and Sam Greydanus developed Dissipative Hamiltonian Neural Networks. These models use a Helmholtz decomposition to learn the conservative energy and the dissipative friction separately, allowing for accurate long-term tracking in environments with air resistance or mechanical damping.
The frontier of this research is now moving from discrete particles to continuous fields. Anthony Zhou and Amir Barati Farimani recently introduced Hamiltonian Neural Solvers (HNS), which treat entire physical fields (like fluids or waves) as infinite-dimensional Hamiltonian systems. By calculating functional derivatives of a learned energy functional, these solvers maintain stability in complex 2D simulations where standard PDE solvers might crash.
A Physical Foundation for General AI
The implications of this research extend beyond physics simulations. By treating the flow of information itself as a physical process, researchers are applying these principles to deep learning architectures. Simon Heilig and Davide Bacciu use Port-Hamiltonian systems to prevent information decay in massive graphs, while Eli Gal and Moshe Eliasof use Hamiltonian dynamics to build reversible Large Language Models that can reconstruct their own internal states, drastically reducing the memory required for training.
By moving from "learning to mimic" to "learning to conserve," this research direction provides a path toward AI that is not just faster, but fundamentally more robust and grounded in the laws of our universe.