When a robot carries out a multi-step manipulation plan, such as stacking twelve blocks into a wall, something small usually goes wrong partway through. A block slips in the gripper, or a neighbor gets nudged. Current systems respond in one of two crude ways. Some keep executing the original plan open-loop, which is why a factored world model that reached 70% success on novel structures in simulation fell to 40% on a real arm (Factored World Models for Zero-Shot Generalization in Robotic Manipulation). Others notice that the world looks different from what they predicted and throw the whole plan away (ATLAS). We propose a middle path. The robot's model of the world is split into per-object pieces, and each skill it runs is a discrete choice ("grasp," "push," "place") plus continuous settings (where, at what angle). When a prediction fails, the robot works out which object's prediction failed and by how much. It then fixes only what is needed: it nudges the continuous settings of the current skill, swaps in a different skill, or re-plans only the steps that touch the affected objects.
The part that makes this work, and the part missing from prior systems, is knowing how large a prediction error is normal for each object. We get that from domain randomization, which means training in simulation across many randomly varied physics settings so that the result transfers to real hardware without real-world training. We randomize the physics object by object rather than all at once. The model then learns a separate spread of expected error for each object, and a real-world surprise can be read as "this block, beyond its normal noise" instead of "something, somewhere."
Core Claim
A factored world model trained under per-factor domain randomization provides calibrated per-object surprise scores. These scores can drive a three-tier recovery policy over a hybrid action space, and that policy recovers from execution failures more often and more cheaply than two alternatives: global-discrepancy replanning, and step-by-step reactive primitive selection. The expected gain is largest on long-horizon, zero-shot sim-to-real assembly, where compounding low-level errors currently dominate.
Research Program
We start from the object-factored graph-network world model of FWM. That model encodes each detected object as a latent vector and predicts the next latents under a continuous pick-and-place action. We make two changes to its action input.
The first change replaces the pick-and-place action with a hybrid action. A hybrid action is a discrete primitive together with its continuous parameters, following the parameterized-action formulation $A = \bigcup_{d \in D} \{d\} \times X_d$ used in CHPO. Primitives terminate on force thresholds, as in the insertion primitives of Learning Insertion Primitives with Discrete-Continuous Hybrid Action Space for Robotic Assembly Tasks. We also borrow the MAPLE-style primitive library of MAPLE.
The second change makes the model predict, for every object, both a next latent and a scale of expected error. Given a primitive $d$ with parameters $x$ applied in state $s$, the surprise for object $i$ is
$$r_i = \frac{\lVert z_i^{\text{obs}} - \hat z_i(s, d, x) \rVert}{\sigma_i(s, d, x)},$$
where $z_i^{\text{obs}}$ is the observed latent of object $i$ after execution, $\hat z_i$ is the model's prediction, and $\sigma_i$ is the predicted error scale for that object.
The $\sigma_i$ are learned under structured randomization. Friction, mass, pose noise, and grasp offset are sampled independently per object, so each object's spread reflects its own uncertainty rather than a global average. This addresses the complaint in the factored-MDP literature that flat randomization "varies all physical parameters simultaneously" and ignores the local-parents structure formalized by Boutilier et al.. We are not claiming the sample-efficiency gains proven in Near-optimal Reinforcement Learning in Factored MDPs. We use factoring for diagnosis, not for regret.
The recovery policy acts on the set of objects whose surprise exceeds a threshold, and it has three tiers.
In the first tier, no object is surprising, so execution continues and no planning compute is spent.
In the second tier, only the primitive's target object is surprising, and only moderately. The system re-optimizes the continuous parameters $x$ with a short cross-entropy search through the model. This is the "temporary failure" tier that Failure Handling In a Planning Framework handled with hand-written rules. Here it comes from a learned model.
In the third tier, the surprise is large or involves objects other than the target. The system first tries alternative primitives on the affected objects. If none reaches the subgoal, it re-plans only the plan suffix whose steps involve the surprising objects or objects they touch in the model's interaction graph. The rest of the plan is kept.
Tier thresholds are set on held-out simulated rollouts with injected failures and then frozen before any real-robot trial. We will also report compute per episode. BRACE showed that unbudgeted replanning can itself cause failures, and localized replanning should use less compute.
We build the system in robosuite (robosuite). The tasks are block assembly analogous to the CUBES domain (towers, walls, stairs) and a peg-insertion stage. Real transfer uses a Franka Panda or UR5 arm, depending on availability. We inject scripted disturbances: a displaced block, reduced gripper friction, and an added mass.
Baselines
The naive reference point is open-loop execution. The factored model plans the full hybrid-primitive sequence and executes it without monitoring. On real hardware the closest reported figure is 40% zero-shot success on the UR5 for the pick-and-place FWM, with 70% on novel CUBES structures and 95% on training structures in simulation (FWM). Our open-loop baseline uses the same model and planner as the full method, so the only difference is monitoring.
The strongest competitor for primitive-level manipulation is MAPLE (MAPLE). MAPLE is a two-level Soft Actor-Critic policy that re-selects a primitive and its parameters at every step, so it is already closed-loop. It reports 90% average success over 8 robosuite tasks, 93% real stacking, and 83% real cleanup. We retrain MAPLE on our tasks with the same primitive library, the same per-object randomization, and the same simulator sample budget. Beating it means higher success on long-horizon assembly under injected disturbances, at matched real-robot trial counts, with non-overlapping confidence intervals. We expect no advantage on short tasks and will report it if there is none.
Two ablations isolate the key ingredients. The first uses the same model and recovery tiers but triggers on global discrepancy, meaning the whole-scene prediction error with a single threshold, followed by full replanning. This is an ATLAS-style trigger ported to physical manipulation; ATLAS itself reports 63.0% on WebArena-Lite (ATLAS), a result that does not transfer as a number here. The second keeps factor attribution but trains $\sigma_i$ under uniform global randomization, which tests whether per-factor randomization is what makes the attribution trustworthy. A third ablation removes the parameter-retune tier, so every surprise goes straight to primitive switching.
Why not just…?
Why not just use a reactive hybrid policy like MAPLE, which already re-decides every step? This is the most serious threat. For short tasks it may be all that is needed. The case for explicit plans is long-horizon structure: a reactive policy has to learn wall-building as a single policy, while a model-based planner generalizes zero-shot to structures it never trained on, as FWM did. The head-to-head on held-out structures settles this. If MAPLE matches us there, the claim reduces to a diagnostic tool, and we will say so.
Why not just retry the failed primitive or replan globally whenever anything looks off? Retrying ignores whether the failure lies in the parameters or in the choice of skill. Global replanning discards valid plan steps, increases compute, and in the setting studied by BRACE caused frequent deadline violations. The global-trigger ablation measures both success and compute directly. Because we inject disturbances with known targets, we can also score whether each method blamed the right object.
Why not just train an end-to-end diffusion policy on demonstrations? Diffusion Policy reports 95% on Push-T and recovers from perturbations within the tasks it was trained on. It needs demonstrations for every task configuration, though, and it has no mechanism for zero-shot assembly of novel structures or for force-terminated insertion. We therefore treat it as out of scope for the zero-shot question rather than as a direct baseline. We would include it only if demonstration data for the held-out structures were available, which would defeat the purpose of testing zero-shot generalization.
Proposed Experiments
The first check is attribution accuracy, and it runs before anything else. In simulation we perturb one known object's physics mid-execution and measure how often the object with the highest surprise is the perturbed one. We compare per-factor $\sigma_i$ against globally trained $\sigma_i$ and against raw unnormalized residuals. If per-factor randomization does not clearly improve attribution, the central mechanism fails and we stop.
The second experiment measures task success under disturbances in robosuite on training and held-out structures. We compare the full method, open-loop execution, MAPLE, the global-trigger ablation, and the no-retune ablation. We report success, number of primitives executed, and planning compute per episode.
The third experiment is zero-shot real transfer. We run fixed trial counts per structure, with the same scripted disturbances physically applied. The target is to close a substantial part of the 70%-to-40% sim-to-real drop reported for open-loop FWM, while matching or exceeding MAPLE's reported real stacking rate on the tasks the two systems share.
Finally, we audit the recovery decisions. We record which tier fired and whether a retune, when attempted, succeeded before escalation. This checks that the tiers are used as intended rather than every failure collapsing into full replanning.
References
- Factored World Models for Zero-Shot Generalization in Robotic Manipulation
- ATLAS: Actor-Critic Task-Completion with Look-ahead Action Simulation
- CHPO: Constrained Hybrid-action Policy Optimization for Reinforcement Learning
- Learning Insertion Primitives with Discrete-Continuous Hybrid Action Space for Robotic Assembly Tasks
- Augmenting Reinforcement Learning with Behavior Primitives for Diverse Manipulation Tasks
- Near-optimal Reinforcement Learning in Factored MDPs
- Failure Handling In a Planning Framework
- When Replanning Becomes the Bottleneck: Budgeted Replanning for Embodied Agents
- robosuite: A Modular Simulation Framework and Benchmark for Robot Learning
- Diffusion Policy: Visuomotor Policy Learning via Action Diffusion