Learning Lagrangians by Minimizing the Action: A Bilevel View of Variational Integrator Training

Variational integrators are simulators built from a discrete version of the principle of least action rather than from the equations of motion. Because of this construction they conserve energy and momentum over very long horizons, and neural versions of them learn mechanical systems from position data alone. Training them well is the hard part. Fitting a learned Lagrangian (the network that encodes kinetic minus potential energy) to multi-step trajectories normally works like this: at every time step, Newton's method solves an implicit equation for the next position, which requires inverting a matrix of second derivatives of the network, and the training gradient must then pass back through every one of these inversions. The cheaper alternative in current use scores how badly observed data violates the discrete equations. That alternative is biased under sensor noise, needs second derivatives of the network, and needs an explicit determinant penalty so the network cannot collapse to a trivial Lagrangian.

We propose to take the physics literally. Hamilton's principle says a trajectory between two fixed endpoints minimizes the action, not merely makes it stationary, provided the time window is shorter than the first "conjugate point." Roughly, that is half the fastest oscillation period. On such short windows, fitting a Lagrangian becomes an ordinary bilevel optimization problem. The inner problem minimizes the discrete action over interior positions with the observed endpoints held fixed. The outer problem fits the resulting path to data. This matters because the inner problem is now strongly convex for a physical reason, so first-order bilevel algorithms developed in machine learning apply with their assumptions actually satisfied. The result would be to train a variational integrator network using only first derivatives of the network, with no Newton solves, no inverse Hessians, and no log-determinant regularizer. The model deployed at test time is still an exact variational integrator.

Core Claim

Let $L_{d,\theta}(q_k, q_{k+1})$ be a learned discrete Lagrangian with parameters $\theta$. For a window of $W$ steps with observed endpoints $\hat q_0, \hat q_W$, write $S_\theta(q) = \sum_{k=0}^{W-1} L_{d,\theta}(q_k, q_{k+1})$ for the discrete action as a function of the free interior positions $q = (q_1,\dots,q_{W-1})$. We train by solving

$$\min_\theta \sum_{\text{windows}} \sum_{k=1}^{W-1} \| q_k^*(\theta) - \hat q_k \|^2 \quad \text{s.t.} \quad q^*(\theta) = \arg\min_q S_\theta(q).$$

Setting the gradient of $S_\theta$ with respect to each interior $q_k$ to zero gives exactly the discrete Euler–Lagrange equations, so any inner minimizer is a genuine trajectory of the learned integrator. We parameterize the Lagrangian as kinetic energy with a Cholesky-factored mass matrix bounded below by $\epsilon I$, minus a neural potential. For small time step $\Delta t$, the action Hessian then behaves like $M/\Delta t$ times a positive-definite tridiagonal matrix, perturbed by potential curvature. It stays positive definite roughly when

$$W \Delta t < \pi / \omega_{\max},$$

where $\omega_{\max}$ is the largest local oscillation frequency of the system. This is the discrete analogue of the Jacobi conjugate-point condition for a harmonic oscillator, whose first conjugate point sits at half a period. Under this condition, the lower-level strong convexity that first-order bilevel theory assumes is a checkable property of the physics. The same mass-matrix lower bound that secures it also rules out the degenerate solution $L_\theta \equiv 0$. Under that degenerate solution every path would minimize the action, and the problem would collapse.

We solve the problem with the value-function approach of BOME! Bilevel Optimization Made Easy. BOME replaces the inner argmin with the constraint $S_\theta(q) \le S_\theta^*$ and approximates $S_\theta^*$ with a few inner gradient steps, then takes a barrier-controlled first-order step on $(\theta, q)$. Every quantity involved is a gradient of the action with respect to $q$ or $\theta$. Those gradients need only first derivatives of the network, because the discrete Lagrangian depends on velocity through finite differences. Per-iteration cost is linear in both window length and degrees of freedom $d$.

Relation to Prior Work

Two papers motivate the problem and constrain the design. Training Structured Mechanical Models by Minimizing Discrete Euler-Lagrange Residual avoids the implicit solve by penalizing the squared discrete Euler–Lagrange residual on observed triplets. Learning mechanical systems from real-world data using discrete forced Lagrangian dynamics does the same with forcing terms. Both need a determinant-based regularizer to keep the Hessian invertible, and Menda et al. note that second derivatives of the network make the method costlier than regression and that scaling to humanoids is open. In system-identification terms, their loss is an "equation-error" objective, because it asks noisy data to satisfy the physics exactly. That is why Kalman pre-smoothing was needed under 0.4 rad noise. Our $W=2$ case is the matching "output-error" objective: the interior point is inferred, not observed, and longer windows average noise across several observations. The contrast is concrete: they differentiate a residual twice, while we minimize an action once.

The training architectures we reuse come from Variational Integrator Networks for Physically Structured Embeddings and Forced Variational Integrator Networks for Prediction and Control of Mechanical Systems. These train through forward rollouts with implicit steps, and our method keeps their deployed model unchanged. On the bilevel side, the Penalty Method for Inversion-Free Deep Bilevel Optimization supplies an alternative first-order solver (a gradient-norm penalty) that we use as a comparison. Efficient Bilevel Optimization with KFAC-Based Hypergradients warns that discarding curvature fails on ill-conditioned inner problems. We take that warning seriously, and it is why the conjugate-point window matters: it is our mechanism for keeping the inner problem well-conditioned rather than hoping it is.

The closest connecting work poses physics inside bilevel problems but differentiates by the implicit function theorem. Dojo does this through an interior-point contact solver, and Single-Level Differentiable Contact Simulation collapses a collision/dynamics hierarchy into one level. Neither treats the variational principle itself as the lower-level optimization. Geometric Methods for Adjoint Systems shows that adjoints of variational integrators inherit symplectic structure, and it raises a concern: do Hessian-free approximations damage that structure? Our design answers the concern by construction. Approximation enters only in which Lagrangian is learned. The deployed integrator is exactly variational, so energy and momentum behaviour can be measured rather than assumed.

Research Program

The work has three parts. First, we establish the conjugate-point condition for the discrete action of the structured Lagrangian family. We derive a sufficient bound on $W\Delta t$ in terms of the mass lower bound and a potential-curvature bound. We then verify it empirically by tracking the smallest eigenvalue of the action Hessian during training, using Lanczos on small systems. This tells a practitioner how long windows can be, and it predicts where first-order methods should start to fail.

Second, we build the trainer. It uses BOME with $T$ inner gradient steps on the window action, warm-started from the previous iterate, and windows sampled from long trajectories. An adaptive rule shrinks $W$ when the Hessian's smallest eigenvalue estimate, obtained from inner-loop progress, approaches zero.

Third, we extend the method to forced and dissipative systems. The Lagrange–d'Alembert equations are not the stationarity conditions of any scalar, so the value-function trick does not apply directly. For this case we fall back to the residual-norm penalty of Mehra and Hamm on free latent interior points, which is still output-error and first-order in the inner loop. We report honestly how much of the advantage survives. Lie-group extensions following Lie Group Forced Variational Integrator Networks are left for later work.

Baselines

The naive reference point is a black-box residual network predicting the next state, trained by one-step regression. Forced Variational Integrator Networks reports that this baseline reaches a 12% pendulum swing-up success rate under model-predictive control, against 100% for FVIN with 5 training trajectories. We rerun this setting to anchor our control numbers.

The strongest competitor on accuracy is a rollout-trained variational integrator network. It uses Newton steps forward and exact implicit differentiation backward, with a structured solver for the per-step linear systems. It uses the exact curvature, so it sets the accuracy ceiling. It is also the method whose cost grows cubically in $d$.

The strongest existing inversion-avoiding method is the discrete Euler–Lagrange residual of Menda et al. with its log-determinant barrier. The file gives no reported number for it on Lacuna, since its results appear only in a figure. We therefore reimplement it, both with and without Kalman smoothing.

A curvature-using hypergradient on our own reformulation is the ablation that isolates our key ingredient. It computes the implicit-function hypergradient for the window action with conjugate gradient on Hessian-vector products. This separates the benefit of the reformulation from the benefit of going first-order.

A second ablation removes the conjugate-point window: we train with $W\Delta t$ well beyond $\pi/\omega_{\max}$ and with no mass lower bound. This tests whether the physics condition, not the solver, carries the result.

All methods use the same architecture, the same trajectories, a matched wall-clock budget per run, and the same hyperparameter search budget (twenty configurations each). We beat the strongest baseline if our long-horizon rollout error and bounded energy error come within 10% of the Newton-and-implicit-differentiation model, at lower wall-clock and memory for $d \ge 50$. We beat the residual baseline if we win outright under noise.

Why not just…?

Why not just unroll the integrator and backpropagate with autodiff? Unrolling keeps the Newton solves in the forward pass, so memory grows with rollout length and the solver iterations must be stored. Dojo documents exploding and noisy gradients from unrolled simulation, and PRDP finds differentiating through iterative solver steps prohibitive without truncation. We include unrolled training at matched wall-clock as a curve in the scaling experiment, so this claim is tested rather than assumed.

Why not just use the residual loss of Menda et al., which already avoids implicit solves? It does avoid them. However, it asks noisy observations to satisfy the physics exactly, needs second derivatives of the network, and needs a log-determinant that costs $O(d^3)$. The noise sweep below is designed to show the bias directly. If the residual method with Kalman smoothing matches us at all noise levels, the remaining advantage is only cost, and we will report exactly that.

Why not just keep implicit differentiation and use conjugate gradient on the window action? This is the real threat. Short windows make the action Hessian well-conditioned, so conjugate gradient may converge in a handful of Hessian-vector products. That alternative is included as an ablation for this reason. Its per-iteration cost is a small multiple of ours, because each Hessian-vector product requires double backpropagation. If it matches BOME at equal wall-clock, the paper's contribution narrows to the windowed action reformulation and its conjugate-point condition. That reformulation makes any hypergradient solver well-behaved, and it would still be the new idea. The first-order solver would become a convenience.

Why not just approximate the inverse Hessian by the identity? On the window action, the inverse Hessian scales with $\Delta t\, M^{-1}$ and is far from the identity. Mehra and Hamm report that this style of approximation (ApproxGrad) reached 68.08% versus 79.03% for their penalty method on CIFAR-10 with 50% label noise. We include the identity approximation in the ablation suite.

Proposed Experiments

The accuracy experiments use the systems where variational integrator networks are established: the undamped pendulum, the double pendulum, and, for the forced extension, the damped pendulum. We measure configuration error over 400-step rollouts, relative energy error over long horizons, and swing-up success under model-predictive control, reproducing the FVIN protocol.

The noise experiment repeats the Menda et al. setting, with Gaussian joint noise up to 0.4 rad, with and without Kalman smoothing. It sweeps window length $W$ and tests the prediction that output-error training degrades more gracefully than equation-error training.

The scaling experiment is the decisive check. It uses a synthetic mass-spring chain with $d$ from 2 to 1000 and known ground truth. We report wall-clock per epoch, peak memory, and final rollout error for all baselines. We expect our method's cost to grow linearly in $d$ and the exact-curvature baseline's to grow cubically. The crossover point is the headline result.

The window experiment sweeps $W\Delta t$ through the predicted bound $\pi/\omega_{\max}$ on the chain and the double pendulum. Inner-loop convergence and final accuracy should break down near the bound. If they break down far from it, our explanation of why the method works is wrong.

Finally, we fit the CMU Graphics Lab motion-capture sequences used by DFLNN, which have roughly 50 or more degrees of freedom. This tests whether the method trains a Lagrangian at a scale where log-determinant regularization and Newton-based rollout training become costly. No reported numerical result was found on Lacuna for this dataset, so all methods are rerun under the shared protocol.