Energy Conserving Neural Architectures for Physical Simulation

28 concepts25 papers1 proposal

When deep neural networks model physical systems over time, they inevitably fall prey to a fundamental flaw: they leak or accumulate energy. If you train a standard Neural Ordinary Differential Equation (Neural ODE) to simulate a simple, frictionless pendulum, the predicted trajectory will not trace a perfect, repeating loop in phase space. Instead, it will slowly spiral outward (gaining unphysical energy) or collapse inward (losing energy).

This happens because standard neural networks are unconstrained black boxes. They attempt to learn the mapping from a current state to its time derivative directly. Even a tiny approximation error at step $t$ compounds at step $t+1$. When rolled out over thousands of timesteps, these errors accumulate into catastrophic divergence.

To fix this, early physics-informed machine learning models introduced "soft constraints"—adding a penalty term to the loss function that punishes the network when it violates conservation laws. However, as Kookjin Lee et al. demonstrated in Structure-preserving Sparse Identification of Nonlinear Dynamics for Data-driven Modeling, soft penalties fail during long-term forecasting. The optimizer routinely trades off physical consistency to minimize data-fitting error, leaving the model vulnerable to numerical drift when extrapolating beyond the training data.

The Hamiltonian Inductive Bias

The breakthrough idea that solves this problem is to enforce conservation laws as a hard architectural constraint. Instead of forcing the network to learn the complex vector field of motion directly, we force it to learn a single, scalar energy function.

In Hamiltonian Neural Networks, Samuel James Greydanus et al. restructured the network's output layer. Given the system's coordinates $q$ (positions) and $p$ (momenta), the network outputs a single scalar $H(q, p)$, representing the total energy (the Hamiltonian). The actual trajectories are then derived analytically using automatic differentiation to satisfy Hamilton's equations:

$$\dot{q} = \frac{\partial H}{\partial p}, \quad \dot{p} = -\frac{\partial H}{\partial q}$$

Why does this simple architectural change guarantee stability? Mathematically, the time derivative of the total energy is:

$$\frac{dH}{dt} = \frac{\partial H}{\partial q}\dot{q} + \frac{\partial H}{\partial p}\dot{p} = \frac{\partial H}{\partial q}\left(\frac{\partial H}{\partial p}\right) + \frac{\partial H}{\partial p}\left(-\frac{\partial H}{\partial q}\right) = 0$$

By construction, the vector field generated by the network is always orthogonal to the gradient of the energy function. The system is physically incapable of drifting to a higher or lower energy state because the network lacks the mathematical degrees of freedom to express a non-conservative trajectory. As Yusuke Tanaka showed in Machine Learning That Reproduces Physical Phenomena from Data, this structural derivation guarantees that the simulation maintains constant energy over time, even when trained on sparse or noisy data.

Symplectic Integration and Phase Space Volume

Learning a perfect Hamiltonian vector field is only half the battle. If we pass this conservative vector field into a standard numerical solver (like explicit Euler or Runge-Kutta), the discretization process itself introduces numerical drift, causing the simulation to degrade over long time horizons.

To preserve the geometric integrity of the system, the neural network must be paired with a symplectic integrator. Symplectic integrators are numerical solvers designed to preserve phase-space volume (satisfying Liouville's theorem). In Sparse Symplectically Integrated Neural Networks (SSINNs), Daniel M. DiPietro et al. integrated high-order symplectic solvers directly into the learning loop. This ensures that the discrete steps taken by the simulation exactly preserve a nearby "shadow Hamiltonian," preventing long-term orbital decay.

Taking this further, Elena Celledoni et al. introduced Universal Energy-Behavior-Preserving Integrators (UEPI), which use neural networks to tune classical discrete gradient methods. This architecture provides rigorous mathematical guarantees of energy preservation regardless of the neural network's training quality.

Scaling to Complex, Dissipative, and Continuous Systems

While basic Hamiltonian networks excel at simple particle systems, real-world engineering demands modeling complex geometries, energy dissipation (friction), and continuous fields.

For complex, multi-body systems where kinetic and potential energy are inextricably linked, Shiying Xiong developed Nonseparable Symplectic Neural Networks, which use an augmented phase space to enforce energy conservation. When dealing with articulated rigid bodies like chains or trusses, Ravinder Bhattoo et al. demonstrated in Learning Articulated Rigid Body Dynamics with Lagrangian Graph Neural Network that embedding Lagrangian mechanics into graph neural networks enables "zero-shot" generalization, allowing a model trained on a 4-link chain to accurately simulate a 100-link chain without accumulating error.

To handle irreversible, dissipative processes without losing structural guarantees, researchers have expanded beyond pure Hamiltonians to broader algebraic frameworks. In Machine learning structure preserving brackets for forecasting irreversible processes, Kookjin Lee et al. introduced GNODE, an architecture that embeds the metriplectic brackets of the GENERIC formalism. This ensures that energy conservation and entropy growth are satisfied simultaneously by design.

For mesh-based physics, the authors of Mesh Field Theory: Port–Hamiltonian Formulation of Mesh-Based Physics developed MeshFT-Net, hard-wiring a port-Hamiltonian structure that splits fixed topology from learnable metrics, guaranteeing that energy is either conserved or dissipated strictly according to physical laws.

Finally, this paradigm scales to infinite-dimensional continuous fields (PDEs). In Hamiltonian Neural PDE Solvers through Functional Approximation, Anthony Zhou et al. extended Hamiltonian mechanics to continuous fields by representing the Hamiltonian as a learnable functional and using functional derivatives to generate physics-preserving neural PDE solvers.

Key Foundational Work

Hamiltonian Neural Networks (2019) — contributed 1 concept elements to this direction.
Sparse Symplectically Integrated Neural Networks (2020) — contributed 1 concept elements to this direction.
Symplectic ODE-Net: Learning Hamiltonian Dynamics with Control (2020) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_hamiltonian_inductive_bias_acfa1847,
  title = {Hamiltonian inductive bias},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/hamiltonian-inductive-bias/rdr_acfa1847bed043b0b8fe2a05e3501f03},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_hamiltonian_inductive_bias_acfa1847_v1,
  title = {Hamiltonian inductive bias},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/hamiltonian-inductive-bias/rdr_acfa1847bed043b0b8fe2a05e3501f03/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake