Trajectory optimizers for drones, spacecraft and robot arms solve a large nonconvex problem every time the start pose, goal or obstacles change. Whether they converge quickly, slowly, or not at all depends heavily on the initial guess. The usual fix is to train a neural network on past solutions to predict a good guess. That works well in distribution, but on unfamiliar obstacle layouts it can do worse than no guess at all. We propose to train the network on two things: what the optimal trajectory is, and how that trajectory bends when the task changes. That second quantity can be computed exactly and cheaply from the solver's own optimality conditions. We also represent rotations so that this "how it bends" quantity stays finite everywhere. Euler angles break down at 90° pitch, and quaternions carry a redundant sign and norm, so we use rotations directly as matrices. A short convergence test after the first solver iterations tells the system when to discard a bad guess.
The obvious combination of these directions would be "differentiate through the solver's optimality conditions to train the warm-start network end to end." It does not work as stated. Inside a basin of attraction, the converged solution does not depend on where the solver started, so its derivative with respect to the initial guess is zero. That is why the only working solver-aware warm-start method, Learning to Warm-Start Fixed-Point Optimization Algorithms, unrolls a fixed number of solver steps instead of differentiating the fixed point. The implicit function theorem is useful for a different job. It gives the derivative of the solution with respect to the task parameters. That derivative is a training target, and it costs only a few extra triangular solves with a matrix the solver has already factored.
Core Claim
Our claim has three parts. First, supervising a warm-start network with both solutions and their parameter sensitivities, computed by implicit differentiation, needs fewer expensive solver calls than plain regression to reach a given iteration count. Second, it degrades more gracefully as the problem moves away from the training data. Third, both benefits depend on the rotation representation: they are reliable on the rotation group itself and unreliable with Euler angles or quaternions.
Concretely, let $\theta$ be the task parameters (start and goal poses, obstacle positions). Let $z^*(\theta)$ be the optimal state–control trajectory, and let $\lambda^*(\theta)$ be its Lagrange multipliers, the solver's internal prices on each constraint. The solver stops when a residual $r(z,\lambda;\theta)$ of the optimality conditions is zero. Differentiating that condition at the solution gives the sensitivity system
$$K(z^*,\lambda^*;\theta)\begin{bmatrix} \mathrm{d}z^*/\mathrm{d}\theta \\ \mathrm{d}\lambda^*/\mathrm{d}\theta \end{bmatrix} = -\,\partial_\theta r ,$$
where $K$ is the solver's Newton (KKT) matrix. On the rotation group, $K$ is written in tangent-space coordinates. The warm-start network $\hat y_\phi(\theta)$, with weights $\phi$, outputs a primal-dual trajectory. It is trained with
$$\mathcal{L}(\phi) = d\big(\hat y_\phi(\theta), y^*(\theta)\big)^2 + \beta\,\big\| D_\theta \hat y_\phi(\theta) - D_\theta y^*(\theta) \big\|_F^2 .$$
Here $y^*=(z^*,\lambda^*)$, $d$ combines a geodesic distance on rotations with a Euclidean distance on positions, velocities and multipliers, $D_\theta$ is the Jacobian with respect to $\theta$, and $\beta$ weights the sensitivity term. Each training sample then supplies a first-order model of the solution map rather than a single point. That is why we expect gains in sample efficiency and near-distribution extrapolation, and little gain far from the data.
Research Program
The solver substrate is the Riemannian direct transcription of Riemannian Direct Trajectory Optimization of Rigid Bodies on Matrix Lie Groups. That work keeps orientations as rotation matrices and discretizes dynamics with a Lie group variational integrator, an integrator that steps directly on the rotation group. It solves with a Riemannian interior-point method whose Newton matrix is banded, so a Newton step costs time linear in horizon length. It supplies our forward solver and our second derivatives. It does not differentiate its solutions or study initialization, and those are the two things we add.
The first piece we build is the backward pass: the sensitivity system above, in left-trivialized tangent coordinates. It reuses the solver's final banded factorization, so each parameter direction costs one linear-time back-solve. Efficient Riemannian Meta-Optimization by Implicit Differentiation shows that implicit differentiation on manifolds works, cutting memory from growth with inner-loop length to constant. It does so only for unconstrained machine-learning objectives, with Neumann-series inverses. We instead have hard equality and inequality constraints and an exact sparse factorization.
Sensitivities can be invalid. Myriad reports that implicit differentiation of trajectory optimizers was numerically unstable and retreated to partial unrolling. We treat that report as a constraint on our design. Our hypothesis is that the instability comes from two sources. One is constraints that are only weakly active, where the solution map has a kink. The other is coordinate singularities of the kind Representation Issues in the ML Estimation of Camera Motion identified as artificial sensitivity spikes. To guard against both, we log the condition number of $K$ and the margin between multipliers and constraint slacks. Samples where a constraint is nearly active get only the value loss.
The second piece is the network output. Following the critique in Deep Learning Warm Starts for Trajectory Optimization on the International Space Station, the network does not output ambient quaternion polynomials, which leave the unit sphere between knots. It outputs tangent-vector corrections to a geodesic interpolation between start and goal, mapped onto the group with the exponential map. Every warm start is therefore a valid rotation trajectory. The network also predicts multipliers. Interior-point methods are notoriously hard to warm-start, so we treat the initial barrier parameter as a tuned hyperparameter and include it in the ablations.
The third piece is a safeguard against bad guesses. After two solver iterations from the warm start, we compute the residual contraction ratio. If it is above a threshold calibrated on validation data, we restart from the cold initialization. The goal is a bounded worst case: never more than about two iterations worse than a cold start, in contrast to the out-of-distribution regressions reported for the Astrobee warm-starter. Provable accelerated gradient method for nonconvex low rank optimization shows basins shrink as conditioning worsens, which motivates a test computed per problem rather than a fixed trust radius.
Baselines
The naive reference point is a cold start. It uses spherical-linear interpolation of orientation and straight-line interpolation of position between start and goal, with zero multipliers, solved by the same Riemannian interior-point solver. Every method's iteration count and failure rate are reported relative to this.
The strongest existing warm-starting approach for rotational trajectory optimization is supervised regression, as in Deep Learning Warm Starts for Trajectory Optimization on the International Space Station and its precursor Learning-based Warm-Starting for Fast Sequential Convex Programming and Trajectory Optimization. There, a multilayer network maps boundary conditions and obstacles to a polynomial trajectory. Reported numbers are a 60% reduction in iterations of the GuSTO solver (sequential convex programming) on rotational maneuvers and 50% on in-distribution obstacle avoidance (Banerjee et al.). Cauligi et al. report a 17% average reduction and about 57% on hard cases. These figures come from a different solver, so we do not compare against them directly. We re-implement the regression baseline twice: once in its original quaternion form, and once with our manifold output head and our solver. Architecture, optimizer and hyperparameter search budget match ours. Beating it means fewer median and 95th-percentile iterations at equal offline compute. Offline compute counts both solver calls and sensitivity back-solves, so regression gets extra samples equal to the cost of our sensitivities.
The strongest solver-aware baseline is unrolled training in the style of Sambharya et al. Here the network is trained through $k$ unrolled interior-point iterations to minimize the residual after those iterations. On a quadcopter MPC task, they report tracking success with 15 OSQP iterations, a budget at which cold starts failed, and 10x to 100x faster arrival at target accuracy. We run it with $k$ in 1–5, matched wall-clock training time, and the same network.
Two further references matter. A nearest-neighbor lookup returns the closest cached solution. A first-order "tangent predictor" moves that cached solution along its stored sensitivity. The predictor uses our sensitivities with no learning at all. If it matches the network, the contribution is the sensitivities, not the training scheme, and we would report it that way.
The key ablation sets $\beta=0$ with everything else fixed. A second ablation swaps the rotation-matrix formulation for Euler angles and for quaternions with norm constraints, keeping the loss fixed.
Why not just…?
Why not just train the regression network on more data? With unlimited solver calls, plain regression may close the gap, and we will not claim otherwise. The claim is about efficiency per expensive solve and about behavior near the edge of the data. The data-budget sweep in the experiments settles this. It plots iterations against offline compute for $\beta=0$ and $\beta>0$. If the curves cross early, the sensitivity term is not worth it.
Why not just unroll a few solver steps, as Sambharya et al. do? This is the real threat. Unrolling optimizes exactly what we care about, downstream solver effort, while sensitivity matching only optimizes a surrogate. Our case rests on three points. Interior-point steps over long collocation grids make unrolling costly in memory, as Investigating Bi-Level Optimization for Learning and Vision from a Unified Perspective: A Survey and Beyond documents for unrolled solvers generally. Short unrolls only see the local region near each start. Our sensitivity targets reuse an existing factorization. If unrolling with $k\le5$ wins at matched training time, we would recommend it combined with our manifold head and fallback test.
Why not just use quaternions with renormalization? Teng et al. already report reaching $10^{-14}$ tolerance on drone docking in significantly fewer iterations than quaternion formulations, and about 10x faster than IPOPT on a KUKA arm. Those results concern the forward solve, not sensitivities or warm starts. A Discrete Variational Integrator for Optimal Control Problems on SO(3) notes that the norm constraint adds multipliers and drift. We expect it to add a near-null direction to $K$ along the norm and to create sign ambiguity in learned targets. The representation ablation tests this directly. If quaternion sensitivities turn out well conditioned, the geometric part of the proposal shrinks to an output-head detail.
Why not just reuse the nearest stored solution? In dense, low-dimensional parameter spaces it may be enough. Its tangent-predictor variant is included as a baseline so that the network has to earn its place.
Proposed Experiments
The first experiment is diagnostic. On Teng et al.'s drone docking task, we build a family of maneuvers whose optimal pitch sweeps through 90°. For each, we compute the condition number of $K$ and the sensitivity norm under Euler angles, quaternions and rotation matrices. Euler angles should show spikes at the singular attitude. If rotation matrices stay flat, the Myriad instability is at least partly a coordinate effect.
The second experiment is the main comparison. It runs on drone docking and on an Astrobee-style free-flyer with polygonal keep-out zones. We regenerate the free-flyer data with our solver, since the original GuSTO dataset may not be available, using the JEM-like setup of about 11,000 instances. Training budgets run from 100 to 10,000 solves. We report median and 95th-percentile iterations, failure rate, and wall-clock time, all relative to cold start.
The third experiment shifts the distribution. We hold out obstacle counts and positions and goal attitudes beyond the training range. Here we measure how much each method loses relative to cold start, with and without the contraction-test fallback. Success means our method with fallback never exceeds cold start by more than two iterations, while the regression baseline reproduces the reported regressions.
The fourth experiment controls for geometry. We run the rocket-landing task in Myriad with flat states, to test whether sensitivity matching helps without any rotation geometry.
The decisive check is the $\beta=0$ ablation under matched offline compute. If sensitivity matching gives no reduction in 95th-percentile iterations at any data budget, the central claim is false. The remaining contributions would then be the manifold output head and the fallback test.
References
- Learning to Warm-Start Fixed-Point Optimization Algorithms
- Riemannian Direct Trajectory Optimization of Rigid Bodies on Matrix Lie Groups
- Efficient Riemannian Meta-Optimization by Implicit Differentiation
- Myriad: a real-world testbed to bridge trajectory optimization and deep learning
- Representation Issues in the ML Estimation of Camera Motion
- Deep Learning Warm Starts for Trajectory Optimization on the International Space Station
- Provable accelerated gradient method for nonconvex low rank optimization
- Learning-based Warm-Starting for Fast Sequential Convex Programming and Trajectory Optimization
- Investigating Bi-Level Optimization for Learning and Vision From a Unified Perspective: A Survey and Beyond
- A Discrete Variational Integrator for Optimal Control Problems on SO(3)