Differentiable model predictive control (MPC) treats a trajectory optimizer as a neural-network layer: the network outputs the costs, dynamics, or constraints, a solver computes the optimal action sequence, and gradients pass back through the solver's optimality conditions. Recent work has made the backward pass cheap. It now costs time linear in the planning horizon $T$, and it parallelizes on GPUs. The forward solve, however, still runs at the robot's native control rate. A task that needs a thousand primitive steps of lookahead therefore needs a thousand-step optimization problem at every training step and every control step. Temporal abstraction offers a fix: plan over a few large "jumps" instead of many small steps. The obvious way to combine the two ideas is to fit a coarse model of the dynamics to predict where the system will be after each jump, and then plan with it. That is unlikely to work. A model that aggregates many steps into one cannot be accurate, and the planner will exploit its errors.
The proposal is to train the coarse model for decisions rather than for prediction. We build a two-level optimization layer. A long-horizon coarse problem over $K \ll T$ learned jumps chooses a waypoint. A short fine-resolution MPC with hard constraints then tracks that waypoint. Both problems are differentiated jointly through their optimality conditions, so the coarse model's parameters are trained by the closed-loop task loss. The coarse model is allowed to be wrong about the future in exactly the ways that do not change good decisions. The goal is near full-horizon control quality at roughly $O(K+H)$ solver cost instead of $O(T)$, where $H$ is the short fine horizon. If the method works, the coarse plan should also stay valid when constraints change at test time, which a learned terminal-value network cannot offer.
Core Claim
Inside an optimization layer, a temporally abstracted dynamics model trained end-to-end through implicit differentiation should beat the same model trained by multi-step regression, at equal model capacity and data. It should also roughly match a full primitive-resolution differentiable MPC at a fraction of the forward and backward cost. The new element is not hierarchy alone. It is the use of decision-focused training, the predict-and-optimize idea from operations research, to learn the jumpy model that a temporal-abstraction planner relies on.
Positioning
The proposal reuses its backward-pass machinery from three papers. Differentiable MPC for End-to-end Planning and Control differentiates an iterative LQR solver (a standard linearize-and-solve trajectory optimizer) through the optimality conditions of its converged trajectory, with backward cost independent of the number of solver iterations. Revisiting Implicit Differentiation for Learning Problems in Optimal Control (IDOC) shows that the differentiated KKT system is block-tridiagonal, solvable in $O(T)$ time, and stable under active thrust constraints where Safe-PDP fails. Differentiable Model Predictive Control on the GPU (DiffMPC) parallelizes the solve across time steps with a preconditioned conjugate-gradient method. All three make the per-step cost cheaper. None shortens the horizon itself, and all three plan only at the primitive control rate. The forward pass at large $T$ is the cost we target.
Two papers motivate the proposal. Compositional Planning with Jumpy World Models (CompPlan) learns multi-step "jumpy" prediction models whose long jumps are trained to agree with chains of shorter jumps. Planning with them by random shooting reaches nearly 80% on AntMaze-Giant, where the base policies reach 0%. CompPlan's model is trained purely for prediction, its planner uses no gradients, and it offers no hard constraints. Action-Sufficient Goal Representations supplies the warning that drives our core claim. Subgoal representations trained to predict value collapse states that need opposite actions, which drops HIQL to 1% on cube-quadruple. Training through the low-level actor instead restores 28%. We read this as evidence that abstractions must be trained against the controller that consumes them. Our controller is an optimizer, so the training signal must pass through its optimality conditions.
The predict-and-optimize literature supplies the training principle. Learning to Reason: Predict-then-Optimize vs Predict-and-Optimize and the survey End-to-End Constrained Optimization Learning: A Survey cover the static-allocation case. Smart Predict-and-Optimize for Hard Combinatorial Optimization Problems shows that training through a cheaper relaxed problem can match regret on the full problem while running orders of magnitude faster. That result is the static analogue of our bet that a coarse-time problem can stand in for a fine-time one during training. Two papers constrain the design. Surrogates such as Landscape Surrogate and SO-EBM drift outside their training support, so we keep an exact constrained fine solver in the loop rather than replacing it.
The Layer
Let $x_t$ and $u_t$ be the primitive state and control, with known or learned one-step dynamics $f$. The coarse problem chooses macro-parameters $z_1,\dots,z_K$, each summarizing $\Delta$ primitive steps. It uses learned jumpy dynamics $F_\theta$ and a learned coarse cost $c_\theta$:
$$\hat{x}^{*}(\theta) = \arg\min_{\hat{x}, z} \sum_{k=1}^{K} c_\theta(\hat{x}_k, z_k) \quad \text{s.t.} \quad \hat{x}_{k+1} = F_\theta(\hat{x}_k, z_k),\ \hat{x}_1 = x_0 .$$
The fine problem is an ordinary hard-constrained MPC over $H \approx \Delta$ steps, with terminal cost $\|x_H - \hat{x}^{*}_2(\theta)\|^2_P$. The constraints include actuator limits and obstacle constraints at every primitive step, so the coarse level never needs to certify feasibility. The outer loss $\mathcal{L}(\theta)$ is either the closed-loop task cost or an imitation loss on executed actions. The gradient $d\mathcal{L}/d\theta$ is obtained by applying the implicit function theorem to the stacked KKT systems of both problems. The theorem says the solution's sensitivity to parameters can be read off the linearized optimality conditions at the solution, without unrolling the solver.
The stacked system has two block-tridiagonal pieces, one of length $K$ and one of length $H$. They are coupled only through the waypoint $\hat{x}_2$. A Schur-complement elimination of that coupling keeps the solve at $O(K+H)$, extending IDOC's structural argument to two levels. This coupled differential system is a modest but real technical contribution.
Research Program
The work proceeds in three parts.
The first part builds the two-level layer and its implicit backward pass in JAX. The pass reuses the forward factorizations, as DiffMPC does. We check it against finite differences and against fully unrolled differentiation of the same layer. We also characterize the cases where the implicit function theorem assumption breaks: inexact coarse solves and changes in which fine constraints are active. We measure gradient error as a function of solver tolerance, because the file flags both failure modes as known weaknesses of implicit gradients.
The second part trains $F_\theta$ and $c_\theta$ under three regimes from the same offline data. The first regime is regression only, with multi-step prediction loss plus CompPlan-style horizon consistency. The second is decision-focused only. The third is regression pretraining followed by decision-focused fine-tuning. This comparison isolates the core claim.
The third part sweeps the jump length $\Delta$ and the number of jumps $K$ at fixed total lookahead $K\Delta = T$. This traces the trade-off between quality and compute, and locates where the regression-trained model's aggregation error starts to hurt decisions.
Baselines
The quality ceiling is full primitive-resolution differentiable MPC over the whole horizon $T$, implemented with DiffMPC and with IDOC-style $O(T)$ implicit gradients. It is the strongest existing method because it optimizes the true fine problem and differentiates it as efficiently as any published approach. The file reports DiffMPC as 4–7x faster than trajax and as reaching imitation-learning convergence 2x faster in wall-clock time than the best baselines. It reports IDOC at about 2x faster gradient computation than PDP. We run it with the same dynamics, constraints, outer loss, data, and hyperparameter search budget as our method. Beating it means matching its task cost within a few percent, or matching its success rate, while using several times less wall-clock time per training step and per control step at $T$ in the thousands.
The do-nothing reference is the fine MPC with only the short horizon $H$ and a hand-set terminal cost toward the goal. It shows how much long-horizon reasoning matters on each task.
The structured but unlearned reference is move-blocking, a classical MPC trick that shortens the problem by holding controls constant over blocks. It is run on the true physics model with blocks of length $\Delta$ and covers the whole horizon $T$ at reduced cost. It shows what temporal coarsening achieves without any learned coarse model.
The key ablation is the identical two-level layer with $F_\theta$ trained only by regression and horizon consistency. Architecture, data, and parameter count are matched. Beating this ablation by a clear margin in task cost is the result that would establish the paper.
A planning-style competitor follows CompPlan: random shooting over the same regression-trained jumpy model, feeding waypoints to the same fine MPC. Sample budgets are set so that wall-clock time matches our layer.
The final baseline is the short fine MPC with a learned terminal-value network trained end-to-end through the same implicit gradients. It is described under the strongest alternative below.
Why not just…?
Why not just use a coarser time step, or move-blocking, in DiffMPC? This needs no learning. Where the physics are smooth, it may well be enough, and the move-blocking baseline exists to detect that. Its weakness is that block-constant controls over long jumps cannot represent long jumps that each need a carefully timed sequence of controls, such as a pendulum swing-up within one jump. We expect the gap to grow with $\Delta$. If move-blocking matches our layer across the $\Delta$ sweep, the learned coarse model is unnecessary and we will report that.
Why not just fit the jumpy model by regression, as CompPlan does, and plan with it? This is the regression-only ablation. The cube-quadruple collapse reported in Action-Sufficient Goal Representations suggests that abstractions fit to a proxy target can discard what the controller needs. The SPO results in Smart Predict-and-Optimize for Hard Combinatorial Optimization Problems show that the decision-focused loss is what makes a cheaper model usable. Regression could still win on harder tasks, because decision-focused gradients through a non-convex coarse problem may be noisy. The pretrain-then-fine-tune regime is our hedge against that outcome.
Why not just learn a terminal-value network on the short fine MPC? This is the most serious threat. A value network trained end-to-end through the fine KKT system is cheaper still, at $O(H)$. On a fixed task it may match us. The case for an explicit coarse optimization is that its constraints and goal can be changed at test time and the problem re-solved, while a value network has absorbed one maze layout and one goal distribution into its weights. We settle this with a held-out split. At test time we add or move obstacles and change goals. The coarse problem receives the new constraints, and the value network is used unchanged. If the value network keeps pace on this split, we will concede that the coarse layer adds only cost.
Why not just unroll both solvers for a few iterations? Amos et al. show that unrolled backward cost grows linearly with iteration count while the implicit pass stays flat. We nonetheless include unrolled differentiation of our own layer, both to validate gradients and to measure the memory crossover point on our tasks.
Proposed Experiments
The control suite extends the tasks used by IDOC and DiffMPC: cartpole, pendulum swing-up, 6-DoF quadrotor maneuvering, and 6-DoF rocket landing with thrust limits. We lengthen their horizons to $T$ between 500 and 5000 primitive steps by adding waypoint sequences and obstacle fields that require long lookahead. For these lengthened tasks the file reports no prior numbers, so all methods are rerun in our harness. The navigation suite is OGBench PointMaze at large and giant scale (OGBench). There the point-mass fine dynamics are known, and the coarse model is learned from the offline data. We found no reported result on Lacuna for PointMaze success rates of HIQL, CRL, or CompPlan. As a stretch goal, AntMaze-Giant with a learned fine dynamics model would allow direct comparison with CompPlan's reported "nearly 80%" success.
The metrics are task cost or success rate, the rate of constraint violations at the primitive level, wall-clock time per forward solve, per backward pass, and to convergence, peak GPU memory, and gradient error against finite differences.
Three results are decisive. The first is the gap between decision-focused and regression-trained coarse models at matched capacity across the $\Delta$ sweep. We predict the gap widens with $\Delta$, because aggregation error grows with jump length. The second is the curve of quality against compute relative to full-horizon DiffMPC. The third is the test-time constraint-change split against the terminal-value baseline. If the first gap is absent, the core claim fails, regardless of the speedups.
References
- Differentiable MPC for End-to-end Planning and Control
- Revisiting Implicit Differentiation for Learning Problems in Optimal Control
- Differentiable Model Predictive Control on the GPU
- Compositional Planning with Jumpy World Models
- Action-Sufficient Goal Representations
- Learning to Reason: Predict-then-Optimize vs Predict-and-Optimize
- End-to-End Constrained Optimization Learning: A Survey
- Smart Predict-and-Optimize for Hard Combinatorial Optimization Problems
- Landscape Surrogate: Learning Decision Losses for Mathematical Optimization Under Partial Information
- End-to-end Stochastic Optimization with Energy-based Model
- OGBench: Benchmarking Offline Goal-Conditioned RL