Adaptive Hamiltonian Variational Integrators and Symplectic Accelerated\n Optimization

Valentin Duruisseaux, Jérémy Schmitt, Melvin Leok

Abstract

It is well known that symplectic integrators lose their near energy preservation properties when variable step sizes are used. The most common approach to combine adaptive step sizes and symplectic integrators involves the Poincaré transformation of the original Hamiltonian. In this article, we provide a framework for the construction of variational integrators using the Poincaré transformation. Since the transformed Hamiltonian is typically degenerate, the use of Hamiltonian variational integrators based on Type II or Type III generating functions is required instead of the more traditional Lagrangian variational integrators based on Type I generating functions. Error analysis is provided and numerical tests based on the Taylor variational integrator approach of Schmitt, Shingel, Leok (2018) to time-adaptive variational integration of Kepler's 2-Body problem are presented. Finally, we use our adaptive framework together with the variational approach to accelerated optimization presented in Wibisono, Wilson, Jordan (2016) to design efficient variational and non-variational explicit integrators for symplectic accelerated optimization.

Introduction: The Challenge of Adaptive Symplectic Integration

In numerical physics, symplectic integrators are the gold standard for simulating dynamical systems, such as orbiting planets, because they preserve the geometric properties (phase space area) of the system. This preservation leads to exceptional long-term energy conservation. However, these integrators face a significant limitation: their energy-preserving properties typically break down when variable time-step sizes are introduced. In many scenarios, such as a planet approaching a sun in a highly eccentric orbit, changing the time-step is essential for maintaining accuracy and efficiency.

To address this, researchers often employ a Poincaré transformation, a mathematical technique that redefines time. While effective for adaptivity, this transformation results in a "degenerate" (singular) Hamiltonian. Traditional Lagrangian-based (Type I) variational integrators cannot handle such degeneracy. This paper introduces a unified framework that utilizes Type II and Type III Hamiltonian Variational Integrators (HVIs) to overcome this hurdle, enabling stable, adaptive, and symplectic numerical integration.

Methodology

The proposed framework integrates three key mathematical components:

  1. Poincaré Transformation: A "monitor function" $g(q, p)$ is introduced to relate physical time $t$ to a "fictive time" $ au$ via $dt/d\tau = g$. By integrating with a constant step in fictive time, the system effectively utilizes a variable step in physical time.
  2. Type II & III Hamiltonian Variational Integrators (HVIs): The core innovation lies in using Type II and Type III generating functions. Unlike standard Type I integrators, these can accommodate the degenerate Hamiltonians produced by the Poincaré transformation.
  3. Bregman Hamiltonians for Optimization: To apply these tools to optimization, the authors map objective functions to physical systems using the Bregman framework. This treats the process of finding a function's minimum as simulating a particle's trajectory in a potential well.

The authors specifically develop Hamiltonian Taylor Variational Integrators (HTVIs). These use Taylor expansions to approximate the system's flow, resulting in high-order accuracy and computational efficiency. The specific algorithmic steps for Type II and Type III HTVIs are detailed below.

Algorithmic structure of Type II and Type III Hamiltonian Taylor Variational Integrators (HTVI).

Key Insights

  • Addressing Degeneracy: The realization that Poincaré-transformed Hamiltonians are almost always degenerate explains the failure of previous Lagrangian attempts at adaptivity. Switching to Type II/III HVIs is the critical solution.
  • Explicit Updates: The authors demonstrate that Type II HTVIs, when paired with specific quadrature rules (like the rectangular rule), result in explicit update equations. This is vital for high-dimensional optimization where solving implicit equations at each step is computationally prohibitive.

Applications and Results

The framework was evaluated on two distinct benchmarks:

1. Kepler’s 2-Body Problem

A classic test in celestial mechanics involving a planet in a highly eccentric orbit ($e = 0.9$ and $0.99$). Such orbits require very small steps near the periapsis (closest approach) and larger steps elsewhere.

  • Result: The adaptive HTVI achieved a global error of approximately $10^{-6}$ using significantly fewer steps than the fixed-step Symplectic Euler-B method. For the $0.99$ eccentricity case, the adaptive method was orders of magnitude more efficient.

2. High-Dimensional Optimization

The authors tested a 50-dimensional quartic convex function: $f(x) = [(x-1)^T \Sigma (x-1)]^2$.

  • Result: The Adaptive HTVI significantly outperformed standard optimization algorithms like Nesterov’s Accelerated Gradient (NAG), ADAM, and RMSprop.
  • Efficiency: To reach a precision of $10^{-10}$, the Adaptive HTVI required only 5,564 iterations, whereas NAG required over 10 billion.
  • Stability: HTVI methods remained stable and converged for non-autonomous systems where traditional fixed-step methods failed.

Conclusion and Future Directions

This work bridges the gap between geometric numerical integration and accelerated optimization. By leveraging Type II and III HVIs within a Poincaré-transformed framework, the authors provide a robust solution for adaptive symplectic integration.

While powerful, the method's performance depends on the selection of the monitor function $g(q, p)$, which currently requires domain expertise. Additionally, high-order Taylor methods involve higher-order derivatives, which can be a bottleneck for extremely large-scale problems despite the aid of Automatic Differentiation. Future research directions include extending this framework to Lie Groups and Riemannian Manifolds for constrained optimization and developing stochastic variants to handle noisy gradients in deep learning environments.

Summary updated

Abstract

It is well known that symplectic integrators lose their near energy\npreservation properties when variable step sizes are used. The most common\napproach to combine adaptive step sizes and symplectic integrators involves the\nPoincar\\'e transformation of the original Hamiltonian. In this article, we\nprovide a framework for the construction of variational integrators using the\nPoincar\\'e transformation. Since the transformed Hamiltonian is typically\ndegenerate, the use of Hamiltonian variational integrators based on Type II or\nType III generating functions is required instead of the more traditional\nLagrangian variational integrators based on Type I generating functions. Error\nanalysis is provided and numerical tests based on the Taylor variational\nintegrator approach of Schmitt, Shingel, Leok (2018) to time-adaptive\nvariational integration of Kepler's 2-Body problem are presented. Finally, we\nuse our adaptive framework together with the variational approach to\naccelerated optimization presented in Wibisono, Wilson, Jordan (2016) to design\nefficient variational and non-variational explicit integrators for symplectic\naccelerated optimization.\n
BibTeX
@article{duruisseaux2021adaptive,
  author = {Valentin Duruisseaux and Jérémy Schmitt and Melvin Leok},
  title = {{Adaptive Hamiltonian Variational Integrators and Symplectic Accelerated\textbackslash\{\}n Optimization}},
  journal = {SIAM J. Sci. Comput},
  year = {2021},
  doi = {10.1137/20m1383835},
}
Figure