Approximation of nearly-periodic symplectic maps via structure-preserving neural networks

Valentin Duruisseaux, J. W. Burby, Qi Tang

TL;DR

This paper introduces "symplectic gyroceptrons," a novel neural network architecture designed to accurately approximate nearly-periodic symplectic maps. These networks are structure-preserving, ensuring both symplecticity and the emergence of a discrete-time adiabatic invariant, crucial for long-term stability in non-dissipative dynamical systems.

What Problem Does This Paper Address?

Predicting the long-term behavior of complex dynamical systems, especially those governed by Hamiltonian mechanics, is computationally challenging. Traditional numerical integration methods can be prohibitively slow, particularly for systems with multiple timescales or requiring high precision. While deep learning offers powerful function approximation capabilities, standard neural networks often fail to preserve the fundamental symmetries and conservation laws inherent in these physical systems, leading to inaccurate and unstable predictions. This paper specifically addresses the problem of learning accurate and stable surrogate models for "nearly-periodic" Hamiltonian systems, which exhibit a specific type of oscillatory behavior and possess an adiabatic invariant.

Methods

The core innovation is the symplectic Gyroceptron, a neural network architecture built upon the concept of HénonNets. HénonNets are themselves designed to approximate symplectic maps by composing simpler "Hénon-like" maps. The Symplectic Gyroceptron leverages this by decomposing a nearly-periodic symplectic map into three components:

  1. A near-identity symplectic diffeomorphism ($I_\epsilon$): This part learns the slow, averaged dynamics of the system. It's implemented using a "near-identity HénonNet" that ensures $I_0$ is the identity map.
  2. A symplectic circle action ($\Phi_\theta$): This component captures the fast, oscillatory motion. It's a fixed, known structure representing the system's fundamental rotation.
  3. A symplectic diffeomorphism ($\psi$): This acts as a conjugation, aligning the learned dynamics with the known circle action. It's implemented using a standard HénonNet.

The architecture is designed such that the composition of these three parts approximates the target nearly-periodic symplectic map. Crucially, the structure-preserving nature of HénonNets ensures that the resulting Symplectic Gyroceptron is itself symplectic. The paper also demonstrates that this architecture naturally leads to the preservation of a discrete-time adiabatic invariant, a key theoretical property of nearly-periodic systems.

Datasets

The paper does not explicitly list or introduce new benchmark datasets. Instead, it focuses on learning surrogate maps for specific, well-defined nearly-periodic Hamiltonian systems. The "datasets" are generated by simulating these systems using classical numerical integrators (like Runge-Kutta 4) for training purposes. The paper presents numerical experiments on two example systems:

  • Nonlinearly coupled oscillators: A system of two coupled oscillators with different frequencies.
  • Charged particle interacting with its self-generated electromagnetic field: A higher-dimensional system modeling a charged particle interacting with its own electromagnetic field.

Key Insights and Ideas

  • Structure Preservation is Key: The paper strongly emphasizes that for learning physical systems, neural networks must preserve fundamental properties like symplecticity. Standard networks fail because they don't inherently respect these constraints.
  • Decomposition for Approximation: By decomposing a complex nearly-periodic symplectic map into a near-identity map, a known circle action, and a conjugation diffeomorphism, the problem becomes tractable for neural networks.
  • Adiabatic Invariant Emergence: A significant finding is that the proposed Symplectic Gyroceptron architecture provably leads to the existence of a discrete-time adiabatic invariant, regardless of the specific weights or biases. This is a crucial theoretical guarantee for long-term stability.
  • HénonNets as Building Blocks: The paper highlights the utility of HénonNets as effective and invertible components for building symplectic neural networks.
  • Bridging Continuous and Discrete Time: The work connects continuous-time nearly-periodic systems (with adiabatic invariants) to their discrete-time analogues (nearly-periodic maps) and shows how neural networks can bridge this gap.

Key Results

  • Accurate Surrogate Models: The Symplectic Gyroceptron successfully learned surrogate maps for both example systems, closely matching reference trajectories generated by high-precision classical integrators.
  • Long-Term Stability: The learned models demonstrated excellent long-term stability, evidenced by the conservation of the adiabatic invariant over many iterations, especially for small values of the parameter $\epsilon$.
  • Computational Efficiency: The Symplectic Gyroceptron achieved significant speedups compared to traditional integrators. For instance, one example showed a speedup of several orders of magnitude.
  • Provable Adiabatic Invariant: The architecture guarantees the existence of an adiabatic invariant, a property not easily achieved by standard neural networks.
  • Comparison with Other Architectures: The Symplectic Gyroceptron outperformed or matched other symplectic neural network approaches (like SympNets) in terms of accuracy and speed, while offering the crucial theoretical guarantee of adiabatic invariant preservation.

Limitations and Future Work

  • Known Circle Action: The current framework assumes the topological type of the limiting circle action is known a priori. Learning this action from data remains an open challenge.
  • Non-resonant Systems: The theory and architecture are primarily developed for non-resonant systems. Extending to resonant systems is a future direction.
  • Hyperparameter Optimization: While the models performed well, the authors suggest that further hyperparameter tuning could potentially improve training outcomes.
  • Data Mining for Model Discovery: The authors propose that Symplectic Gyroceptrons could potentially be used for discovering dynamical models from observational data, but this requires developing data-mining methods for learning the topological conjugacy class of the limiting circle action.
  • Generalization to Multiple Oscillations: While the framework can be generalized to handle multiple non-resonant modes, it cannot be applied to systems exhibiting resonant surfaces.

Summary updated

Abstract

A continuous-time dynamical system with parameter [Formula: see text] is nearly-periodic if all its trajectories are periodic with nowhere-vanishing angular frequency as [Formula: see text] approaches 0. Nearly-periodic maps are discrete-time analogues of nearly-periodic systems, defined as parameter-dependent diffeomorphisms that limit to rotations along a circle action, and they admit formal U(1) symmetries to all orders when the limiting rotation is non-resonant. For Hamiltonian nearly-periodic maps on exact presymplectic manifolds, the formal U(1) symmetry gives rise to a discrete-time adiabatic invariant. In this paper, we construct a novel structure-preserving neural network to approximate nearly-periodic symplectic maps. This neural network architecture, which we call symplectic gyroceptron, ensures that the resulting surrogate map is nearly-periodic and symplectic, and that it gives rise to a discrete-time adiabatic invariant and a long-time stability. This new structure-preserving neural network provides a promising architecture for surrogate modeling of non-dissipative dynamical systems that automatically steps over short timescales without introducing spurious instabilities.
BibTeX
@inproceedings{duruisseaux2023approximation,
  author = {Valentin Duruisseaux and J. W. Burby and Qi Tang},
  title = {{Approximation of nearly-periodic symplectic maps via structure-preserving neural networks}},
  booktitle = {Scientific},
  year = {2023},
  doi = {10.1038/s41598-023-34862-w},
}
Figure