A robot running model predictive control (MPC) plans by re-solving a short-horizon trajectory optimization every few milliseconds, using a learned model of its own dynamics. When the robot's physics change, for example through a new payload or actuator wear, that model must be updated online. The update should not erase what the model knew about earlier conditions, since the robot may return to them. On an embedded GPU, the update also runs on the same chip as the planner. Gradient steps and replay-buffer reads compete with the planner for shared cache and memory bandwidth. This is the "noisy neighbor" effect: logically separate jobs slow each other down through hardware they share without knowing it. The two failures are coupled. The usual cure for forgetting, a replay buffer of past transitions, is also the component that generates the most memory traffic. A larger buffer protects old knowledge and makes the planner miss deadlines. A smaller buffer can make forgetting worse.
We propose to judge methods for preventing forgetting on the metric that matters on shared hardware: how much old-regime knowledge they keep per unit of interference they impose on the co-running controller. We also propose a specific method we expect to sit on the best part of that trade-off. The method is replay-free Jacobian distillation. It penalizes changes in the dynamics model's derivatives at a small fixed set of stored anchor points, with the teacher's derivatives precomputed and sized to fit a cache slice. It runs alongside a gate that admits gradient steps only when the planner's measured solve-time margin allows. The bet rests on how nonlinear MPC uses the model. Gradient-based solvers use the model's Jacobians, meaning its local linearizations, more than its raw predictions. So the Jacobians are the part of old knowledge that most needs protecting, and they can be stored compactly.
Core Claim
The claim is that nonlinear MPC with continual model adaptation on a shared accelerator should be evaluated along two axes, forgetting and controller deadline safety. On that pair of axes, compact derivative distillation beats replay and weight-importance regularization. This is a claim about both axes jointly, not about retention alone.
The method draws on three lines of prior work. It borrows its core technique from Continual Learning of Physical Systems via Derivative Distillation. That paper regularizes a physics-informed network on the derivatives of a frozen teacher and reports near-zero forgetting on PDE regression without replay. It stays a passive, offline surrogate model and never closes a control loop or runs under a hardware budget. The claim is motivated by Replay can provably increase forgetting, which proves that in over-parameterized linear regression, replaying a small subset can turn vanishing $O(1/T)$ forgetting into $\Theta(1)$ forgetting. That result means "just use a smaller buffer to reduce traffic" is not a safe way out. The claim is constrained by Assessing the Use of NVIDIA Multi-Instance GPU in the Automotive Domain and iGniter. These show that co-located kernels inflate a victim's execution cycles 2.97x and its L2 misses about 27x, and slow ResNet-50 inference by over 35%. Contention on these chips is large enough to break a millisecond-scale control loop.
The closest systems work comes from control and continual reinforcement learning. Asynchronous Computation of Tube-based Model Predictive Control already splits a fast control thread from a slow background optimization. It assumes linear dynamics and threads that do not interfere in hardware. Continual Model-Based Reinforcement Learning with Hypernetworks and Continual Reinforcement Learning by Planning with Online World Models adapt world models continually for planning, but on isolated hardware. None of this work measures what the learner does to the planner's timing, and that measurement is what we add.
Research Program
The system has a planner thread and an adaptation thread on one GPU. The planner is a nonlinear MPC. We run two variants because they load the hardware differently. One is a sampling-based Cross Entropy Method planner in the style of ABC-LMPC. It uses thousands of parallel rollouts and is exactly the "bully" workload that CIAO describes, meaning it evicts co-runners' cached data. The other is a gradient-based sequential quadratic programming planner that queries model Jacobians.
The dynamics model $f_\theta(x,u)$ predicts the next state from state $x$ and control $u$. When a regime ends, we store a fixed anchor set $\mathcal{A}$ of state-control pairs taken from that regime's nominal MPC trajectories, together with the frozen model's Jacobians $J_{\text{old}}(x,u)$ at those points. The adaptation thread then minimizes
$$ \mathcal{L}(\theta) = \sum_{(x,u,x')\in\mathcal{B}} \|f_\theta(x,u)-x'\|^2 + \lambda \sum_{(x,u)\in\mathcal{A}} \|J_\theta(x,u)-J_{\text{old}}(x,u)\|_F^2 , $$
The terms are defined as follows. $\mathcal{B}$ is a small window of recent transitions from the current regime. $J_\theta$ is the current model's Jacobian with respect to $(x,u)$. $\|\cdot\|_F$ is the Frobenius norm. $\lambda$ weights retention against fitting the current regime.
Because the teacher Jacobians are precomputed, no teacher network is evaluated during training. The retention term's memory footprint is fixed at $|\mathcal{A}|$ times the Jacobian size. It can be chosen to fit the cache share the planner leaves free. This footprint argument is what distinguishes the method from replay: the memory cost of preventing forgetting becomes a design parameter rather than a side effect.
The gate is deliberately simple. Let $d$ be the control deadline and $\hat\tau_t$ the running 99th-percentile planner solve time over the last $N$ cycles. The adaptation thread launches its next step only when
$$ d - \hat\tau_t > \delta , $$
where $\delta$ is a safety margin. The gate uses measured solve time as feedback. We chose this over an offline interference model like iGniter's because the workload mix changes as regimes shift.
The analysis will trace, for each retention method, a frontier of old-regime error against planner deadline-miss rate, sweeping buffer size or anchor count. We will also check the mechanism directly, by testing whether Jacobian error on old regimes predicts closed-loop cost on return to those regimes better than prediction error does. If it does not, the rationale for distilling Jacobians specifically is wrong. In that case we would fall back to output distillation at anchors, which matches the combined OutD+DerD variant of the PDE paper.
Baselines
The naive reference point is naive fine-tuning. The model is adapted with plain gradient steps on the recent window, with no retention mechanism, no gate, and full co-execution with the planner. It shows how much forgetting and how much deadline damage occur when nothing is done.
A second reference point is a non-adaptive robust controller. It is a fixed model trained on all regimes offline, run with constraint tightening in the spirit of the tube methods in Linear robust adaptive model predictive control. It has no contention from learning at all. Asynchronous Computation of Tube-based MPC shows the price of this conservatism in the linear case: control cost 85.92 for standard tube MPC versus 67.36 for its adaptive variant. We expect the gap to be at least as large for nonlinear systems, and we will measure it.
The strongest retention competitor on raw accuracy is output-plus-derivative distillation as published. Continual Learning of Physical Systems via Derivative Distillation reports an error of $8.96 \times 10^{-4}$ on Allen-Cahn with shifting parameters, against $7.65 \times 10^{-3}$ for Elastic Weight Consolidation (EWC). EWC penalizes moving weights that a Fisher-information estimate marks as important. The PDE paper's distillation evaluates a live teacher network at collocation points, so its footprint grows with teacher size.
We will also run reservoir experience replay at several buffer sizes, including buffers that exceed the L2 cache. Replay is the default practice, and the non-monotone buffer-size behavior predicted by Replay can provably increase forgetting may appear in dynamics models too. EWC runs as a buffer-free regularizer.
Every method gets the same regime sequences, the same model architecture, the same per-regime gradient-step budget, and the same hyperparameter search budget, tuned on a held-out regime sequence. All methods run under the same gate so that gating is not confounded with retention.
Two ablations isolate the key ingredients. The first is gate-only: replay under the gate, with no distillation. It tests whether scheduling alone solves the problem. The second is distillation without the gate. It tests whether the small footprint alone suffices.
The strongest systems competitor is hardware partitioning. It is emulated on an A100 with Multi-Instance GPU (MIG), which physically splits the GPU into isolated slices, using two partitions for planner and learner. The MIG paper reports that isolation holds a small victim kernel at 1.01x cycles under heavy contention. It also reports a "partitioning tax": a medium kernel rose from 10.9 million to 38.0 million cycles once confined to its slice. No reported result was found on Lacuna for closed-loop nonlinear MPC with continual adaptation under GPU contention, so we have no published number to beat on the main metric.
Beating the baselines means the following. At matched old-regime error, our method has a lower 99th-percentile solve time and fewer deadline misses than every replay configuration. At matched deadline-miss rate, it has lower old-regime closed-loop cost than EWC and gate-only replay. It also beats MIG partitioning on total closed-loop cost, with no more deadline misses.
Why not just…?
Why not just partition the GPU with MIG, since there are only two workloads? This is the most serious threat. On the A100 it may simply work, because two partitions are enough for one planner and one learner, and measured isolation is near perfect. We therefore treat MIG as a baseline rather than dismissing it. Two costs remain. The first is the partitioning tax: the CEM planner's rollouts will not fit a half-cache slice, much like the medium kernel that went from 10.9M to 38.0M cycles. The second is that the MIG paper notes embedded chips like Orin have only two GPU processing clusters, and that the partitioning mechanism faces architectural barriers there. The experiment settles this directly. If MIG gives lower closed-loop cost at equal deadline safety, the contribution narrows to the retention method, and we will say so.
Why not just give the planner a high-priority stream and run learning whenever the GPU is idle? Priority controls which kernel is dispatched first. It does not stop an already-running backward pass or a replay gather from evicting the planner's cached data, which is where the 27x L2-miss inflation in the MIG study comes from. The gate-only ablation is this alternative with a better trigger. If gate-only replay matches our frontier, the Jacobian-distillation ingredient is unnecessary.
Why not just skip adaptation and use a robust controller over all regimes? This removes contention entirely but pays in conservatism. For nonlinear systems, tightening constraints to cover every regime can leave no feasible trajectory at all. The non-adaptive baseline measures that price directly, in closed-loop cost and task success on the harder regimes.
Why not just use a small replay buffer that fits in cache? The replay theory predicts that small buffers can land exactly in the regime where forgetting gets worse. Our replay sweep will show whether that non-monotone curve appears for dynamics models. If it does not, and small replay matches distillation, replay stays competitive, and the frontier plot will show it.
Proposed Experiments
Tasks come from the ABC-LMPC environments: the noisy inverted pendulum, the 7-link reacher, and point-mass navigation. We add a legged task in MuJoCo Playground, with batched rollouts in the style of Isaac Gym. Physical parameters such as mass, link length, friction, and actuator gain change in cyclic regime sequences like A→B→C→A. The return to A measures forgetting in closed loop.
Hardware is an A100, with and without MIG, plus an embedded Orin-class GPU running unpartitioned. The contention level is calibrated against the vector-copy microbenchmark from the MIG study.
Metrics cover three groups. Timing metrics are planner 99th-percentile solve time and deadline-miss rate. Control metrics are constraint violations and closed-loop cost per regime. Retention metrics are old-regime prediction error and Jacobian error, plus hardware counters for L2 misses.
The first decisive check comes before anything else. We measure planner solve-time inflation under naive co-located adaptation. If deadline inflation is small, for instance within the margin at the task's control rate, the systems half of the claim collapses and the work becomes a closed-loop comparison of retention methods only.
The second check is the frontier comparison against replay, EWC, and MIG described above.
The third check is the mechanism test: whether old-regime Jacobian error predicts return-to-regime closed-loop cost better than prediction error does. This separates "distilling Jacobians helps control" from "any compact regularizer helps."
References
- Continual Learning of Physical Systems via Derivative Distillation
- Replay can provably increase forgetting
- Assessing the Use of NVIDIA Multi-Instance GPU in the Automotive Domain
- iGniter: Interference-Aware GPU Resource Provisioning for Predictable DNN Inference in the Cloud
- Asynchronous Computation of Tube-based Model Predictive Control
- Continual Model-Based Reinforcement Learning with Hypernetworks
- Continual Reinforcement Learning by Planning with Online World Models
- ABC-LMPC: Safe Sample-Based Learning MPC for Stochastic Nonlinear Dynamical Systems with Adjustable Boundary Conditions
- CIAO: Cache Interference-Aware Throughput-Oriented Architecture and Scheduling for GPUs
- Linear robust adaptive model predictive control: Computational complexity and conservatism
- MuJoCo Playground
- Isaac Gym: High Performance GPU Based Physics Simulation For Robot Learning