A robot arm mounted on a drone or a legged body pushes back on its own base every time it moves. When an imitation-learned policy (a network trained to copy human demonstrations) runs on such a robot with a load the demonstrator never handled, such as a heavier payload, its arm motions throw larger reaction forces into the base. The base tilts in ways never seen during training, the onboard camera swings, and the policy now sees inputs it was not trained on. It reacts badly, and the error compounds. Imitation learning calls this failure covariate shift: small errors push the robot into unfamiliar states, where its next actions are worse. On floating bases, a large share of that shift is caused by the policy's own actions and can be predicted from physics before it happens.
We propose a training-free execution layer, which we call a wrench governor, that sits between a pretrained imitation policy and the robot. Before each predicted chunk of actions is executed, the governor computes the force and torque the chunk will exert on the base (the reaction wrench) using the arm's inverse dynamics under the current load estimate. It compares that prediction with the range of reaction wrenches seen in the demonstrations near the current state. If the chunk would leave that range, the governor slows the chunk down in time without changing its path. Slowing a joint trajectory by a factor $s$ divides its inertial reaction by $s^2$ and leaves the geometry of the motion untouched. The policy therefore stays on the spatial path it learned, and the base stays inside the dynamic regime it learned in. The approach also answers a weakness of current whole-body systems, where the manipulation policy never receives feedback about base stability: here the base's dynamic margin directly controls how fast the policy's output is executed.
Core Claim
For an imitation policy on a floating or legged base, much of the deployment-time state drift under changed load comes from the inertial part of arm-to-base coupling. This part can be removed by time-reparameterizing the policy's own action chunks. The gravitational part, the static tilt caused by a shifted center of mass, cannot be removed this way. The physics behind the claim is simple. With the base held fixed and the arm path $q(\tau)$ executed at slowdown $s \ge 1$, so that $q(t) = q(t/s)$, joint velocities scale by $1/s$ and accelerations by $1/s^2$. The predicted base reaction therefore splits as
$$\mathbf{w}_b(s) = \mathbf{w}_g(q) + \frac{1}{s^2}\,\mathbf{w}_{in}(q, \dot q, \ddot q),$$
where $\mathbf{w}_g$ is the configuration- and payload-dependent gravitational wrench and $\mathbf{w}_{in}$ collects the inertial and Coriolis terms of the coupled mass matrix. The governor picks the smallest $s$ for which $\mathbf{w}_b(s)$ lies inside a demonstration-derived envelope. That envelope is the mean $\mu$ and covariance $\Sigma$ of reaction wrenches, computed by inverse dynamics on the $k$ demonstration states nearest the current state:
$$s^\star = \min\{\, s \ge 1 : (\mathbf{w}_b(s)-\mu)^\top \Sigma^{-1} (\mathbf{w}_b(s)-\mu) \le \chi^2 \,\}.$$
This is a scalar search, solvable in closed form along the $1/s^2$ ray, so it runs at control rate.
The novelty claim is specific. Prior floating-base work predicts reaction wrenches so that the base can absorb them. Prior imitation work corrects drift kinematically or in visual-feature space. We use predicted wrenches to stop the imitation policy from generating the shift in the first place, and we measure "in distribution" against the demonstrations' own wrench statistics rather than a hand-set physical limit.
Position Relative to Prior Work
The method's core ingredient comes from Combining Learning-based Locomotion Policy with Model-based Manipulation for Legged Mobile Manipulators. That work has an arm model-predictive planner forecast the next 0.8 s of reaction wrenches, and a reinforcement-learned base policy lean into them. It reports a 57% reduction in end-effector deviation and resistance to 150 N lateral pushes without stepping. We reuse its wrench-forecasting step but invert its direction. There, the base treats arm wrenches as unalterable, which the file flags as an open limitation. Here, the arm's timing yields to the base, and the arm is an imitation policy rather than a model-based planner.
UMI on Legs predicts end-effector targets in the world frame so that a whole-body controller can absorb base motion. Communication in that system flows one way, and it was not evaluated for drift under heavy loads. Our governor adds an upward signal without retraining the policy, and it is compatible with task-frame actions.
AIR-VLA supplies the main testbed and the motivating failure: a quadrotor with a 7-DoF arm in Isaac Sim, where large vision-language-action models show some implicit compensation but collapse when the fixed third-person cameras are removed. We read that collapse partly as base-motion-induced shift in the egocentric images. Our governor attacks the cause, base tilt, rather than the images.
On the imitation side, DART and Grasping with Chopsticks widen the training distribution with noise. KATS synthesizes recovery rollouts with a linear Koopman model, which the file notes struggles with contact discontinuities. Difference-Aware Retrieval Policies conditions actions on offsets to retrieved demonstration states. Error-Aware Imitation Learning from Teleoperation Data for Mobile Manipulation detects drift with a learned reconstruction model and halts. All of these operate on kinematic or visual state, and all except the last change training. We borrow nearest-neighbor retrieval only to build the local wrench envelope, and our response to impending drift is to slow down rather than stop. Toward the Fundamental Limits of Imitation Learning motivates the effort: errors in plain behavior cloning grow quadratically with horizon. Our bet is that, on floating bases, preventing the self-induced part of the per-step error is cheaper than learning to recover from it.
Research Program
The governor needs three components. The first is a payload-aware inverse-dynamics model of the arm on the base. In simulation it uses true inertial parameters. In a second, realistic variant, payload mass and center of mass are estimated online from joint-torque residuals, so we can measure how estimation error degrades the governor.
The second component is the demonstration wrench library. We replay every demonstration through inverse dynamics, store reaction wrenches alongside proprioceptive and base states, and index them for nearest-neighbor lookup in a proprioceptive embedding. Retrieval here only sets an envelope, so modest errors in the distance metric widen or narrow the envelope rather than corrupting actions. This is a deliberate hedge against the metric-distortion problem noted for retrieval policies.
The third component is the time-reparameterization of action chunks. Diffusion Policy and the action-chunking transformer emit multi-step chunks. The governor stretches each chunk by $s^\star$, interpolates it at control rate, and asks the policy for its next chunk at the stretched time. The gravitational term $\mathbf{w}_g$ cannot be reduced by slowing down. If the gravitational term alone leaves the envelope, the governor passes the predicted static offset to the base controller as a feedforward trim. That is the only place we touch the base, and we ablate it.
The analysis is as important as the method. We will decompose deployment drift into a gravitational and an inertial share using the equation above. We will also measure how far deployed base states fall from the demonstration distribution, using Mahalanobis distance on base attitude and angular momentum. This lets us test directly whether failures under load correlate with that distance, and whether the governor reduces it.
Baselines
The naive reference point is the unmodified pretrained policy run under the new loads. We use Diffusion Policy and $\pi_{0.5}$ trained on the AIR-VLA demonstrations, with the stock flight controller. AIR-VLA reports $\pi_{0.5}$ as the strongest overall model and Diffusion Policy as a standard imitation baseline, but the file gives no numeric success rates. No reported result was found on Lacuna for AIR-VLA per-suite success rates, so we will reproduce both under the published protocol before any comparison.
The strongest existing competitor for base stability is predictive base feedforward in the style of Combining Learning-based Locomotion Policy with Model-based Manipulation. The same forecasted wrench sequence our governor computes is fed to the base controller as a disturbance preview, and the arm runs at full speed. This competitor is strong because it addresses the same physics with no loss of task speed, and on legs it achieved the 57% deviation reduction and 150 N push resistance quoted above. For fairness, it receives the identical inverse-dynamics model, payload estimates, and preview horizon, and its feedforward gain is tuned with the same number of evaluation rollouts used to tune the governor's $\chi^2$ threshold. We count the governor as beating it if it achieves higher task success at matched or lower peak base tilt across the heavy-payload conditions. We also test the combination of the two, which we expect to be best, because the two methods act on different terms.
The strongest training-side competitor is load-randomized fine-tuning. The policy is fine-tuned on additional data generated by replaying demonstrations in simulation under randomized payloads, keeping only successful rollouts. This mirrors DART's idea of widening the training distribution, whose reported effect is 79% versus 49% grasping success over behavior cloning on the Toyota HSR. The comparison uses the same simulator budget, measured in environment steps, that we spend building the wrench library and tuning thresholds.
A wrench-observation baseline retrains the policy with predicted base wrench and attitude appended to its inputs. A retrieval baseline uses DARP-augmented Diffusion Policy, the best reported imitation-side drift corrector in the file at 82% versus 52% on Robosuite Stack.
Two ablations isolate the key ingredient. The first is uniform slowdown, where every chunk is stretched by the average $s^\star$ the governor ends up using. This tests whether the benefit comes from state-dependent timing or merely from going slower. The second replaces the demonstration-derived envelope with a fixed physical tilt limit, which tests whether anchoring to the demonstration distribution matters for covariate shift beyond plain safety.
Why not just…?
Why not just train a bigger model on more demonstrations? AIR-VLA's own evidence is that foundation models partially compensate for floating-base jitter but still collapse with egocentric cameras only and show spatial-grounding failures. Demonstrations can also only cover the loads the teleoperator happened to carry. We test this by including $\pi_{0.5}$ as a governed and ungoverned policy. If ungoverned $\pi_{0.5}$ is already flat across payloads, the claim fails for large models, and we will report that.
Why not just let a predictive base controller absorb the wrenches? This is the real threat. On a quadruped with feet on the ground it may be enough. On a quadrotor, however, the base's ability to reject torque is bounded by rotor thrust margins, and a heavier payload shrinks those margins exactly when arm wrenches grow. The head-to-head and combined runs above settle it. The governor matters only if, at high payload, absorption alone saturates and fails while the combination succeeds.
Why not just randomize payload during training? Randomization broadens what the policy has seen but does not bound what it will see. Each new load still produces a new mix of tilt and camera motion. It also costs retraining for every embodiment. The matched-budget comparison measures this. If randomized fine-tuning matches the governor at all tested payloads, the governor's remaining value is only that it is training-free, and we will say so.
Why not just slow the policy down everywhere? Uniform slowdown removes the inertial term too, and it is the ablation most likely to come close. It costs task time everywhere, and it cannot help dynamic tasks such as tossing or swinging motions, where speed is part of success. A tie with uniform slowdown at equal mean task duration would mean state-dependent timing adds nothing.
Proposed Experiments
The primary testbed is the AIR-VLA simulator and its 3,000-episode dataset. We focus on the Base Manipulation and Long-Horizon suites under egocentric-only cameras, which is the regime the file identifies as fragile. Without retraining, we add payloads of increasing mass to the gripper or grasped objects, plus the wind disturbances the benchmark already provides. A secondary legged testbed rebuilds a kettlebell-pushing task in Isaac Gym, in the spirit of UMI on Legs, to check whether conclusions transfer when the ground absorbs part of the load.
The metrics are task success, peak and RMS base attitude error, end-effector tracking deviation, mean task duration, and deployed-state Mahalanobis distance to the demonstration distribution.
Four checks decide the outcome. First, under nominal load the governor should almost never activate and should leave success unchanged; if it costs success there, the envelope is miscalibrated. Second, success under increasing payload should degrade markedly less with the governor than without it, at a duration cost well below uniform slowdown. Third, in a decomposition test we apply heavy but slow-moving loads, where drift is almost entirely gravitational. Here time scaling should provide little benefit and the static trim should provide most of it. If time scaling helps strongly in this regime, our account of the mechanism is wrong. Fourth, across all conditions, distance to the demonstration distribution should predict failure, and the governor's success gain should be explained by its reduction of that distance.
As a stress test we include contact-release tasks such as pulling a plug. In these tasks the released wrench comes from the environment rather than the arm, so we expect limited benefit and report it as the governor's boundary.
References
- Combining Learning-Based Locomotion Policy With Model-Based Manipulation for Legged Mobile Manipulators
- UMI on Legs: Making Manipulation Policies Mobile with Manipulation-Centric Whole-body Controllers
- AIR-VLA: Vision-Language-Action Systems for Aerial Manipulation
- DART: Noise Injection for Robust Imitation Learning
- Grasping with Chopsticks: Combating Covariate Shift in Model-free Imitation Learning for Fine Manipulation
- Koopman-Assisted Trajectory Synthesis: A Data Augmentation Framework for Offline Imitation Learning
- Difference-Aware Retrieval Policies for Imitation Learning
- Error-Aware Imitation Learning from Teleoperation Data for Mobile Manipulation
- Toward the Fundamental Limits of Imitation Learning