SympNets: Intrinsic structure-preserving symplectic networks for identifying Hamiltonian systems

Pengzhan Jin, Zhen Zhang, Aiqing Zhu, Yifa Tang, George Em Karniadakis

Hamiltonian systems, which govern energy-conserving physical phenomena from planetary motion to molecular dynamics, possess an underlying geometric structure known as symplecticity. While standard neural networks often fail to preserve this structure, leading to numerical drift and energy instability, this paper introduces SympNets. These networks are designed to be intrinsically symplectic by construction, offering a robust and efficient framework for learning physical laws from data.

The Challenge of Symplecticity

In classical mechanics, Hamiltonian systems are defined by their conservation of energy and the preservation of symplectic structure in phase space. Traditional deep learning approaches, while powerful, are typically structure-agnostic. When applied to physical simulations, they often suffer from "numerical drift," where the system erroneously gains or loses energy over time. Previous attempts to address this, such as Hamiltonian Neural Networks (HNNs), learn the energy function but rely on external numerical solvers, which can be computationally intensive and struggle with non-separable systems.

Proposed Architecture: SympNets

SympNets shift the focus from learning the Hamiltonian function to learning the symplectic map (the flow) that evolves the system state over time. By designing every layer of the network as a symplectic transformation, the entire architecture is guaranteed to preserve the system's geometric properties. The authors propose two primary variants: LA-SympNets (Linear-Activation) and G-SympNets (Gradient).

Architecture of a SympNet showing the flow from input (p, q) to output (p_n, q_n) through symplectic layers.

A key theoretical contribution is the "Universal Approximation Theorem" for SympNets, proving that these restricted architectures can represent any symplectic map, ensuring they remain as expressive as their structure-agnostic counterparts.

Experimental Evaluation and Efficiency

The models were benchmarked against three systems of increasing complexity: the simple pendulum, the chaotic double pendulum, and the high-dimensional three-body problem. Evaluations were conducted across different data regimes, including regular flow data, large time-step intervals ($h=3.0$), and irregular, randomly sampled data.

SympNets demonstrate significant advantages in long-term stability and generalization. In tests with large time steps where standard HNNs failed, SympNets remained stable. Furthermore, they exhibit remarkable parameter efficiency. For instance, in the pendulum task, an LA-SympNet achieved state-of-the-art performance with only 14 parameters, compared to the 2,000 parameters required by a standard S-HNN.

Comparison of model complexity and parameter counts across different Hamiltonian problems.

Beyond efficiency, SympNets are significantly faster during both training and inference because they avoid the need for gradient computations of a Hamiltonian function during the forward pass. Their analytic reversibility also allows for memory-efficient training, similar to reversible networks (RevNets).

Conclusion

SympNets provide a powerful, "physics-by-design" approach to modeling Hamiltonian systems. By embedding symplecticity into the network architecture, they achieve superior accuracy, stability, and efficiency compared to extrinsic physics-informed models. Future work aims to extend this framework to dissipative systems and explore its potential in generative modeling and stochastic dynamics.

Summary updated

BibTeX
@article{jin2020sympnets,
  author = {Pengzhan Jin and Zhen Zhang and Aiqing Zhu and Yifa Tang and George Em Karniadakis},
  title = {{SympNets: Intrinsic structure-preserving symplectic networks for identifying Hamiltonian systems}},
  journal = {Neural Networks},
  year = {2020},
  doi = {10.1016/j.neunet.2020.08.017},
}
Figure