Several groups now try to make robots and learned controllers robust by tuning them so that nearby trajectories converge. They measure this with Lyapunov exponents, the average exponential rates at which small perturbations grow or shrink along a trajectory. A popular target is the sum of all exponents, which measures how fast a small cloud of states shrinks in volume. The trouble is that the systems where this matters most are stiff: they mix very fast, strongly damped motions with slow ones, as stiff joints, compliant contacts, and damped actuators do. Stiff systems are also exactly where numerical integrators misbehave. A method of nominal accuracy order $p$ often converges at a much lower effective order on stiff problems, which is called order reduction. We argue that when exponents are read off a discrete simulator, the fast modes' exponents mostly reflect the integrator's damping behavior rather than the physics. Those fast modes also dominate the volume-contraction sum. An optimizer that pushes this sum down is therefore partly rewarding the time-stepper, and partly making the system stiffer, while the slow directions that actually decide whether a robot recovers from a push are barely touched.
The proposal is to replace "minimize the sum of all exponents through the simulator" with "minimize the sum of the slow-subspace exponents." These would be computed with variational equations integrated by a collocation method whose internal stage accuracy matches its overall order, so it does not suffer order reduction. We would first show and quantify the integrator artifact on standard stiff benchmarks. We would then show that the slow-subspace objective gives exponent estimates that converge at the integrator's full order as stiffness grows. Finally, we would test whether it produces controllers that are more robust under perturbation than volume-contraction optimization, maximal-exponent penalties, or domain randomization.
Core Claim
Consider a stiff linear mode with eigenvalue $\mu \ll 0$. An integrator with step $h$ maps the perturbation by its stability function $R(h\mu)$. The exponent the simulator reports is $\hat\lambda = h^{-1}\log|R(h\mu)|$, while the true exponent is $\mu$. For L-stable methods such as backward Euler or Radau IIA, $R(z)$ decays only like $1/|z|$ as $z \to -\infty$. In the regime $h|\mu| \gg 1$ that stiff solvers are built for, this gives roughly
$$\hat\lambda \approx -\frac{\log(h|\mu|)}{h}, \qquad \text{versus the true value } \mu = -|\mu|,$$
so the measured contraction depends mainly on $h$ and is far weaker than the true rate. Because the volume sum $L_a = \sum_i \lambda_i$ adds these fast-mode values to the few slow ones, a simulator's $L_a$ is a function of the time step. Its gradient with respect to damping or gains comes from the slope of $R$, not from the physics.
Liouville's formula says the true sum is the long-time average of the divergence of the vector field,
$$L_a = \lim_{T\to\infty}\frac{1}{T}\int_0^T \operatorname{tr} J(x(t))\,dt,$$
where $J$ is the Jacobian of the dynamics along the trajectory $x(t)$. This exposes a second problem. In a stiff system $\operatorname{tr} J$ is dominated by the large negative fast eigenvalues, so even the exact $L_a$ mostly measures how stiff the system is, not how well it rejects disturbances along its slow motion.
Our claim has two parts. The robustness-relevant quantity is $\Lambda_k$, the sum of the $k$ largest exponents, defined on the slow subspace separated from the fast ones by a spectral gap. That quantity can be estimated without order reduction only if the variational equations are integrated by a method of high stage order.
Research Program
The work has three stages.
Stage 1: quantify the artifact. On smooth stiff benchmarks we would compute Lyapunov spectra from discrete-map Jacobians, the approach of Fadini & Coros. That paper differentiates through a rigid-body simulator and takes SVDs of step Jacobians to minimize $L_a$ for co-designing manipulators, spiders, and quadrupeds. We would compute these spectra under semi-implicit Euler, explicit RK4, low-stage-order diagonally implicit Runge-Kutta (DIRK), and Radau IIA. We would sweep both the step size $h$ and the stiffness parameter $\epsilon$, and compare against a reference computed with very small steps at high precision. Two outcomes count as evidence. First, fast exponents and $L_a$ should drift with $h$ in the way the stability-function formula predicts. Second, slow exponents should converge at order $q<p$ under the DIRK and IMEX schemes, mirroring the local-error blow-up in $(h/\epsilon)^k$ analyzed by the splitting local-error study. IMEX schemes treat the stiff part of the equations implicitly and the rest explicitly.
Stage 2: build the estimator. The base trajectory and its tangent dynamics would be integrated with Radau IIA collocation, and with the fully implicit-explicit polynomial block methods of Buvoli & Southworth. Those methods' stage order equals their classical order, and on Van der Pol they keep full convergence order from $\epsilon=1$ to $10^{-8}$. We would separate slow from fast directions using finite-time Lyapunov vectors, following the finite-time Lyapunov analysis of two-timescale systems. That work isolates slow manifolds from the spectral gap and reports invariance errors as low as 0.0001%. However, it relied on standard integrators and hand-tuned windows to discard boundary layers. Only the $k$ slow tangent vectors are propagated and re-orthogonalized, so the QR cost scales with $k$ rather than the full state dimension.
Gradients of $\Lambda_k$ with respect to design and policy parameters would come from the implicit function theorem at each converged collocation step, the technique of implicit single-step training for stiff neural ODEs. That avoids the reverse-time adjoint, which Kim et al. showed is unstable on stiff problems.
Stage 3: use the estimator as an objective. We would optimize damping, stiffness, and gains in co-design problems, and then policies, against $\Lambda_k$ instead of $L_a$, and measure robustness directly. We restrict the main claims to smooth or compliant-contact dynamics. Hard impacts break the polynomial smoothness that collocation relies on, and we treat them only as a stress test.
Baselines
Naive reference. Discrete-map spectra from whatever integrator the simulator already uses, at its default step. This is what current Lyapunov-based robotics does in practice. It is run on identical models and initial conditions, and our estimator must beat it on error against the reference spectrum at equal wall-clock time.
Strongest existing method. Direct $L_a$ minimization through a differentiable simulator, as in Fadini & Coros. That paper reports reducing $L_a$ from −0.56 to −0.61 for a crawling quadruped policy, and improving $L_a$ more than fivefold for a planar manipulator. We would re-run their objective on the same parameterizations with the same optimizer, iteration budget, and horizon. We would then evaluate both their solutions and ours with the reference-accuracy estimator and under external impulses. Beating it means lower perturbation-recovery time and failure rate at matched nominal task cost. We expect our solutions may well have a less negative $L_a$, and we would report that.
Maximal-exponent penalty. The penalty of Young & Pugeault estimates the top exponent from variance across three imagined Dreamer V3 rollouts. It moved Pointmass from chaotic to stable and kept higher reward under observation noise on Walker, but left residual chaos on Walker Run. The file gives no exact reward numbers for it. We would run it at its published sample count and with a matched compute budget, on the same tasks.
Domain randomization. Domain randomization and adversarial training from RRLS are run with the suite's protocol of 10 seeds and the same environment-step budget. No reported result was found on Lacuna for RRLS returns, so we would produce them ourselves.
Ablations. Two ablations isolate the key ingredients. The first keeps Radau IIA and replaces it only by an SDIRK method of equal classical order but lower stage order, which isolates stage order. The second keeps the integrator fixed and swaps $\Lambda_k$ for exact Liouville $L_a$, which isolates the choice of objective.
Why not just…?
Why not just shrink the step size until the spectrum converges? On a 3- to 8-dimensional benchmark this works and is our reference. But the step must fall below $\epsilon$ to leave the error plateau, so cost grows like $1/\epsilon$. That is untenable inside an optimization loop at $\epsilon=10^{-6}$. Stage 1 reports error versus wall-clock time so the reader can see exactly where small-step explicit integration stops being competitive. If it never stops being competitive on our robot models, we will say so.
Why not just compute $L_a$ with Liouville's formula, which only needs the trajectory? This is a real threat to the "fix the estimator" half of the idea. It removes the stability-function artifact for the sum at almost no cost, which is why we include it as an ablation. It does not answer the objective question. The exact $L_a$ is still dominated by the fast-mode trace, so minimizing it still rewards adding stiffness. Stage 3 tests directly whether exact-$L_a$ solutions are less robust than $\Lambda_k$ solutions at matched cost. If they are not, the slow-subspace objective is unnecessary.
Why not just use the top exponent, or regularize stiffness away? The top exponent alone is the cheap special case $k=1$, and on low-dimensional tasks it may suffice. We would sweep $k$ and report when the second and third slow directions matter. Stiffness penalties such as Pal et al. and architectural damping such as Tri-Scale NODEs change the dynamics being modeled. In Tri-Scale NODEs, removing the damping terms raised metabolite error by 9,839%. These approaches are therefore unsuitable when the stiffness is physical, which is the case we target.
Proposed Experiments
Estimator accuracy. The numerical benchmarks are the singularly perturbed Van der Pol oscillator over $\epsilon \in [10^{-8},1]$, the Davis-Skodje system, and the 4D singularly perturbed mass-spring-damper from the finite-time Lyapunov analysis paper, plus ROBER and HIRES. The metrics are absolute error of each exponent and of $\Lambda_k$ against the reference, observed convergence order in $h$ at each $\epsilon$, and the invariance error of the recovered slow subspace. For the last metric, the FTLA paper's reported values (0.0001% versus 5.7% for the earlier ILDM method at one test point) set the bar our boundary-layer-inclusive version must match.
The decisive check for the artifact. For a fixed physical system, the simulator's $L_a$ should change by an amount predicted by $h^{-1}\log|R(h\mu)|$ as $h$ varies, while $\Lambda_k$ from our estimator stays fixed to within reference tolerance.
Optimization and robustness. The tasks are manipulator and quadruped co-design reproduced after Fadini & Coros, and MuJoCo Hopper, Walker2d, and HalfCheetah with damping and mass perturbations from the RRLS suite. The metrics are worst-case and average return under perturbation, recovery time after impulses, and the stiffness ratio of the optimized system. The decisive check for the objective is whether $\Lambda_k$-optimized systems match or beat $L_a$-optimized ones on recovery while ending up measurably less stiff.
References
- Enhancing Robotic System Robustness via Lyapunov Exponent-Based Optimization
- The behaviour of the local error in splitting methods applied to stiff problems
- Additive Polynomial Time Integrators, Part I: Framework and Fully Implicit-Explicit Collocation Methods
- Characterizing two-timescale nonlinear dynamics using finite-time Lyapunov exponents and subspaces
- Training stiff neural ordinary differential equations with implicit single-step methods
- Stiff Neural Ordinary Differential Equations
- Enhancing Robustness in Deep Reinforcement Learning: A Lyapunov Exponent Approach
- Rrls: Robust reinforcement learning suite
- Opening the Blackbox: Accelerating Neural Differential Equations by Regularizing Internal Solver Heuristics
- Tri-Scale Neural ODEs for Continuous Multi-Omics Disease Modeling