Swapping Legs as a Recovery Move for Quadruped Manipulation

Legged robots are starting to use their legs as arms. A quadruped can stand on three legs and press a button, open a door or push a box with the fourth. When that manipulating leg fails, though, current systems have few good options. The foot may slip off a handle, run out of reach, or meet a latch too stiff for one leg's motors. Most systems either keep tracking a target that no longer makes sense or stop. Stopping is dangerous for a robot balanced on three legs with a fourth in the air under load. Yet the robot often already has the fix in its own body: plant the failing leg to restore a four-legged stance, then lift the other front leg and finish the job from the side that has reach, clearance or torque to spare.

This note proposes treating that limb handoff as a learned recovery action. A fast monitor watches the manipulating leg and forecasts, about half a second ahead, whether the current attempt is heading toward failure and why. A short monitor of this kind is a model that predicts upcoming contact and joint-torque trouble from the robot's own sensors and a depth camera. The monitor's answer then picks among three responses: retry with the same leg after a base adjustment, hand the task off to another leg, or pass the problem up to a slow language-model planner. The central claim is narrow on purpose and can be tested. Retrying is enough for transient failures such as a single slip. Handing off pays only for failures tied to a particular limb, such as workspace limits, one-sided blockage, torque saturation or a weakened motor. A monitor that tells these cases apart in under one control cycle should beat both "always retry" and "always switch."

What already exists and what is missing

The multi-limb work supplies the body skills but no recovery. ReLIC, in Versatile Loco-Manipulation through Flexible Interlimb Coordination, assigns each leg a role through a binary mask and blends a model-based reaching controller with a learned locomotion controller. It reaches 78.9% overall success across 12 Spot tasks, but it runs manipulation open-loop and never changes the mask in response to a failure. Visual Manipulation with Legs comes closest to our idea. It scores both front legs with learned "critic maps" and picks the better one, but it makes that choice once, before acting, and its only fallback is a fixed walk-and-reset routine. Pedipulate and Perceptive Pedipulation with Local Obstacle Avoidance train single policies that hold a tripod stance robustly, with 0.057 m hardware foot-tracking error and rejection of 132 N base pushes. Both are hard-wired to the front-right leg, and the second needs an external decision to toggle its contact-switch flag between avoiding and pushing. Learning Visual Quadrupedal Loco-Manipulation from Demonstrations re-plans when a target moves in free space, but it does not handle contact failures and uses a fixed manipulating leg. These papers supply our low-level methods: tripod-stance pedipulation trained in Isaac Gym with workspace curricula. Our contribution is to make the limb assignment change in the middle of a task, driven by a failure forecast.

The error-recovery work motivates the proposal and also constrains it. Conditional Multi-Stage Failure Recovery for Embodied Agents recovers failures through staged language-model reasoning about subgoals, preconditions, workarounds and a final reflection. It reaches 36.46% success on the TEACH household benchmark. Its authors report that the remaining failures are physical: wrong approach angles and missing reach. That is exactly the class a limb handoff addresses. The same multi-second reasoning loop is far too slow for a robot losing balance, so our fast loop cannot be a language model. ContactGuard supplies the monitoring method. A latent world model that predicts future states from planned action chunks scores failure risk in under 20 ms, with ROC AUC 0.992 on its Cup task and 0.946 on Box. However, it only vetoes motions, and only on fixed dual-arm robots. We reuse its predict-then-probe design. We extend it to condition on the limb mask, so the same model can be queried with alternative assignments ("what if the left leg did this?"), and to output a failure class rather than a single stop signal. ReCAPA shows that predictive correction limits cascading errors: its error propagation rate at lag 10 is 0.082, versus about 0.3 for GPT-4o-mini. That supports forecasting before contact rather than reacting afterward. From AgentRefine we borrow one training idea: learn from trajectories that contain a mistake followed by a correction, and weight the loss on the corrective part.

Research Program

The system has three parts, all trained in Isaac Gym and then transferred to a Unitree-class quadruped.

The first part is a mask-conditioned pedipulation policy. It is a single reinforcement-learning policy, in the style of Pedipulate, that takes a foot target and a one-hot choice of manipulating leg (front-left or front-right; rear legs are an optional extension). We add "handoff episodes" that switch the manipulating leg partway through the task. These episodes reward planting the old leg inside the support polygon before the new leg lifts, and penalize base tilt and foot-height spikes during the transition. Following AgentRefine, episodes are deliberately seeded with failures, and the reward weight is concentrated on the steps after the failure.

The second part is a failure forecaster. It is an action-conditioned latent predictor that reads proprioception, joint torques and a depth image, together with the planned action chunk and the limb mask. It predicts the next 0.5 s of foot-contact state and torque-limit margin. A small classification layer on top outputs a failure probability and a class: transient slip, limb-specific (reach, blockage, torque saturation or motor derating), or task-level (for example an object that is truly immovable). Labels come for free in simulation from the failure-injection generator described below.

The third part is a selector running at 50 Hz. When forecast risk under the current mask crosses a threshold, the selector queries the forecaster with counterfactual inputs: the current leg with a base shift and the other leg, plus abort-and-stand as a safe default. It executes the lowest-risk option. If every option stays high-risk, it stands on four legs and escalates to a slow planner in the style of CMFR. Under this design the language model is consulted only after the robot is already stable.

Baselines

The naive reference is the nominal pedipulation policy with no recovery. It keeps tracking its target until timeout or a fall. This measures how much the injected failures hurt.

The second reference is a hand-written fallback state machine, the rule "on contact loss or torque saturation, plant the foot, shift the base toward the target, retry up to three times." It uses the same low-level policy and the same retry budget as our system. Our system beats it if it achieves higher success on limb-specific failures with no worse fall rate on transient ones.

The third baseline is critic-map re-selection, adapted from Visual Manipulation with Legs. We train per-leg success critics on the same data and re-evaluate them at every replanning step, so the robot switches whenever the other leg scores higher. This is the strongest competitor because it already compares the two legs; it only lacks a forecast of failure and a failure class. It receives the same simulation sample budget and threshold tuning as our selector. That paper reports 60% hardware box-flipping success with an initial leg choice. On our failure-injected tasks, no reported result was found on Lacuna, so we run it ourselves.

The fourth baseline is a ContactGuard-style abort-only monitor. It uses our forecaster but answers every predicted failure with stand-and-retry. Comparing it against our selector isolates the value of the counterfactual limb query.

On nominal, failure-free tasks the strongest reported numbers are 92.33% for Press Button and 82.33% for Pulling a Handle, from Learning Visual Quadrupedal Loco-Manipulation from Demonstrations. These set the bar for checking that adding recovery does not degrade nominal performance. Our system should stay within a few points of the nominal policy we train ourselves on these tasks. Two further ablations remove handoff training (switching legs with a policy never trained on mid-task switches) and replace the forecaster's class output with oracle labels, which bounds how much error comes from misdiagnosis.

Why not just…?

Why not just retry with the same leg after moving the base? For slips and small misalignments this probably works as well as anything, and we expect our selector to choose it in those cases. It cannot help when the failure belongs to the limb. Examples are a handle placed outside the right leg's workspace, an obstacle on the right side, or a right-knee motor derated to 60% torque. A base shift then either keeps failing or moves the stance outside the support polygon, which is the risk noted for three-legged balance in Pedipulate. The failure-type breakdown in our experiments settles this directly. If retry matches us on limb-specific failures, the handoff claim is false.

Why not just re-run the leg-selection critics continuously? This is the real threat. Critics trained on success may already prefer the other leg once a failure is under way. Two differences could matter. Our forecaster acts about 0.5 s before contact rather than after progress stalls, and it separates transient failures from limb-specific ones, which should prevent the back-and-forth switching that a pure success score invites on a single slip. If the critic baseline matches our success, fall rate and switch count, the contribution reduces to the handoff-trained low-level policy, and we will report it that way.

Why not just train one end-to-end policy with heavy domain randomization and let it discover switching? The Learning Visual Quadrupedal Loco-Manipulation paper reports that standard hierarchical reinforcement learning reached 21.67% on Press Button and 0% on Pulling a Handle on its suite, so unstructured learning struggles even without injected failures. We still include an end-to-end policy given access to all four legs and our full failure injection, trained with matched compute, so this claim is tested rather than assumed.

Why not just ask a vision-language model what to do? CMFR's own analysis shows such models diagnose semantic gaps but not motor-level problems. OrchDAG (OrchDAG) reports 15 to 24% zero-shot accuracy for frontier models re-planning after dependency failures. Their latency is also measured in seconds, while the recovery has to start within a 20 ms control cycle. The language model stays in our system only as the escalation path after the robot is stable.

Proposed Experiments

We build a failure-injected version of the Isaac Gym suite from He et al. (button, valve, handle, basket, dishwasher), plus box pushing and flipping from Visual Manipulation with Legs. Each episode samples one injected failure from five types: a sudden drop in friction on the contact surface, the target relocated outside the active leg's workspace, an obstacle blocking one side, a latch whose resistance exceeds single-leg torque but not the other leg's margin in that posture, and torque derating of one leg's motors. Failure-free episodes are mixed in to measure regression.

The metrics are task success, fall rate, time from failure onset to resumed progress, repeated identical failed attempts (the robot's version of AgentRefine's "death loops"), and number of leg switches. For the forecaster we also report ROC AUC per class and lead time.

The decisive check is a matrix of success rate by failure type and method. The hypothesis predicts a crossover. Retry should match or beat our system on transient slips. Our system should clearly beat retry and critic re-selection on the three limb-specific types. Nobody should succeed on task-level failures except through escalation. A second decisive check asks whether the counterfactual risk estimates are calibrated: when the forecaster says the other leg would succeed, does a forced switch actually succeed at that rate? If the forecasts are not calibrated, the selector's choices are luck. Finally, we run 20 trials per condition on hardware for the reach, blockage and derated-motor cases, the last implemented by software torque limits. We compare against the retry machine and critic re-selection and report falls separately, because one fall on hardware matters more than a few points of success in simulation.