Neural Hamiltonian Diffusions for Modeling Structured Geometric Dynamics

Sungwoo ParkNeurIPS 2025 · OpenReview

Introduction

Neural Hamiltonian Diffusion (NHD) is a novel framework designed to learn stochastic physical laws on curved surfaces, known as manifolds, rather than being restricted to flat spaces. By "lifting" dynamics to a frame bundle, NHD ensures that the learned physics remains consistent regardless of the chosen coordinate system. This approach significantly outperforms existing models in complex domains like molecular simulations, quantum systems, and relativistic astrophysics.

The Problem with Flat-Space Physics

Most machine learning models for physics—such as Hamiltonian Neural Networks (HNNs)—assume the world operates on a flat, "Euclidean" grid. While this works perfectly for a pendulum swinging on a table, it fails to capture the complex geometries inherent in the real world.

Physical systems frequently evolve on non-Euclidean domains:

  • Molecules: Internal rotations (like dihedral angles in proteins) naturally live on a torus.
  • Quantum Mechanics: Spin states evolve on curved Lie groups.
  • Astrophysics: Gravity bends the fabric of spacetime, creating a curved manifold.

Furthermore, real-world data is rarely pristine; it is filled with thermal fluctuations and observational noise. Existing tools typically handle the geometry (e.g., Riemannian diffusions) but ignore the underlying physics, or they handle the physics (e.g., HNNs) but fail to account for geometry and noise. NHD bridges this gap by unifying stochastic differential equations (SDEs) with Hamiltonian mechanics directly on differentiable manifolds.

Core Methodology

The central innovation of NHD lies in its handling of curvature and coordinate systems.

  • Hamiltonian Diffusion on Manifolds: Instead of predicting a simple deterministic path, NHD models the system as a Stratonovich SDE. The "drift" of this SDE is driven by a neural-network-parameterized Hamiltonian ($H_\theta$) that inherently respects the manifold's Riemannian metric and Poisson structure.
  • The Frame Bundle Lift: To avoid the mathematical mess of coordinate-dependent calculations on a curved surface, the authors "lift" the dynamics to the orthonormal frame bundle. Imagine attaching a local "compass" to every point on the manifold. By ensuring the model is gauge-equivariant, the physics remains identical even if you rotate that local compass.
  • Architecture: The model utilizes a Gauge-Equivariant Transformer U-Net. This architecture integrates canonicalization (transforming coordinates into a local frame) and performs invariant computations, ensuring the learned vector fields are both physically plausible and numerically stable.
Illustration of the frame bundle lift and gauge-equivariant vector fields.

Datasets and Key Results

The authors evaluated NHD across three distinct, highly complex scenarios:

  1. Toroidal Molecular Dynamics: Predicting the trajectories of protein backbones (such as Alanine dipeptide AD-3) where states reside on high-dimensional tori.
  2. Quantum Spin Systems: Modeling interacting bodies on a 3-sphere ($S^3$).
  3. Relativistic N-Body Problems: Simulating particles spiraling toward a Schwarzschild black hole, governed by a curved Lorentzian manifold.
Complex simulated trajectories on curved manifolds, demonstrating the model's ability to capture intricate geometric dynamics.

NHD consistently outperformed state-of-the-art baselines (including EGNN, EqMotion, and GeoTDM) across all tasks:

  • Accuracy: In molecular dynamics (AD-3), NHD achieved an Average Displacement Error (ADE) of 0.023, compared to 0.045 for the next best model and 0.413 for standard HNNs.
  • Stability in Relativity: In the black hole simulation, NHD was the only model to accurately capture the relativistic deflection and inward inspiral of particles. Standard HNNs produced inconsistent, physically impossible trajectories.
  • Energy Drift: NHD exhibited significantly lower cumulative relative energy drift over time compared to Euclidean baselines, proving it successfully learned the underlying physical constraints of the curved space.

Key Insights

  • Curvature as Complexity: The paper establishes a theoretical link between manifold curvature and model generalization. High curvature intensifies "stochastic distortion," meaning models must be geometrically aware to avoid overfitting.
  • Stationarity over Conservation: In traditional HNNs, energy is perfectly conserved ($dH/dt = 0$). In NHD's stochastic environment, energy conservation is redefined through the stationarity of the equilibrium distribution. The Hamiltonian governs long-term statistical behavior rather than a single fixed trajectory.
  • Gauge Equivariance is Essential: Enforcing gauge equivariance (independence from local frame orientation) intrinsically reduces the "worst-case" deviation of the model, effectively eliminating spurious variance caused by arbitrary coordinate choices.

Limitations and Future Work

While groundbreaking, NHD has a few limitations:

  • Scalability: Performance degrades as the number of interacting bodies ($N$) increases. Scaling to massive molecular systems remains a computational challenge.
  • Prior Knowledge: The framework requires knowing the manifold's metric and Poisson structure beforehand. For "black box" systems, these geometric properties would need to be learned alongside the dynamics.
  • Computational Overhead: Integrating SDEs on the frame bundle is more computationally intensive than standard ODE integration in flat space. Optimizing this "lifted" simulation for real-time applications is a critical next step.

Summary updated

Abstract

We introduce Neural Hamiltonian Diffusion, a unified framework for learning stochastic Hamiltonian dynamics on differentiable manifolds. While Hamiltonian Neural Networks (HNNs) model conservative systems in flat Euclidean space, they fail to account for geometric structure and intrinsic stochasticity. Conversely, diffusion models on Riemannian manifolds offer geometry-aware stochastic modeling but lack physical inductive biases. Our method parameterizes a Hamiltonian with a neural network and defines its dynamics as a stochastic differential equation on a (pseudo-)Riemannian manifold equipped with a Poisson structure. This enables physically consistent modeling of dynamics on curved, periodic, or causally structured spaces. We demonstrate that the proposed geometric dynamics generalizes existing approaches and applies to systems ranging from molecular dynamics to relativistic n-body problems.
BibTeX
@inproceedings{park2025neural,
  author = {Sungwoo Park},
  title = {{Neural Hamiltonian Diffusions for Modeling Structured Geometric Dynamics}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2025},
  url = {https://openreview.net/forum?id=VswQY0peMr},
}
Figure