TL;DR
Visual world-model planners fail for two distinct reasons: either the optimizer misses a good action (search failure) or the model incorrectly overrates a bad one (scoring failure). This paper introduces a simple diagnostic probe to distinguish these failure modes and proposes DemoProp, a method that uses retrieved demonstrations to either "warm-start" the search or bypass the model entirely by averaging demonstration windows, significantly improving success rates across diverse robotic tasks.
What Problem Does This Paper Address?
Visual world models (like DINO-WM and JEPA-WM) allow robots to plan complex tasks without explicit rewards by imagining future outcomes and selecting actions that lead to a goal image. However, these planners often fail. Traditionally, researchers respond by simply "searching more" (increasing compute) or "improving the model."
The authors argue that these blanket responses are inefficient because they ignore the root cause of failure. A planner can fail in two fundamentally different ways:
- Search Failure: The model actually knows what a good action looks like, but the optimizer (usually the Cross-Entropy Method, or CEM) simply never sampled it.
- Scoring Failure: The model is overconfident in its own errors. It assigns a lower cost to a failing action than to a successful one, essentially "tricking" the optimizer.
Without knowing which failure occurred, you don't know whether to add more compute or fix the model's overestimation.
Methods
The authors propose a known-solution probe to diagnose failures. By taking a known successful action sequence (from a demonstration) and scoring it with the planner's own cost function, they can compare it to the optimizer's best candidate.
- If the model prefers the known solution but the optimizer didn't find it, it's a search failure.
- If the model prefers its own failing candidate over the known solution, it's a scoring failure.
To fix these, they introduce DemoProp (Demonstration Propagation), which retrieves relevant "windows" of actions from a demonstration library based on the robot's current state and goal.
- In the Search Regime: They use a "warm start," seeding the optimizer with these retrieved demonstrations.
- In the Scoring Regime: Since the model's judgment is untrustworthy, they use "Average"—simply executing the mean of the retrieved demonstration windows, bypassing the model's faulty scoring.
Datasets
The study evaluates three world models across four simulated environments:
- PushT: A 2D planar task involving pushing a T-shaped block to a goal pose.
- Wall: A 2D navigation task requiring an agent to pass through a specific door.
- Metaworld: A 3D manipulation benchmark (e.g., opening a faucet, sliding a plate).
- RoboCasa: A photorealistic kitchen simulator using models trained on real-robot data (DROID dataset).
Key Insights and Ideas
- The Optimizer's Curse: In the scoring regime, searching more can actually be harmful. A larger search is more likely to find "adversarial" actions that exploit the model's prediction errors, leading to even lower predicted costs for actions that fail in reality.
- Environment-Specific Regimes: PushT and Wall are primarily limited by search. Metaworld and RoboCasa are dominated by scoring failures. Interestingly, the cost function is often "sound" on real outcomes (high AUROC) but "at chance" on the model's own noisy predictions.
- The Power of Averaging: When a model is in the scoring regime, the best move is often to ignore its selection entirely. Averaging nearby demonstrations cancels out individual "scatter" in the data without falling into the trap of the model's overestimation.
Key Results
- Search Regime (PushT): Using DemoProp to expand the candidate pool raised the success rate of DINO-WM from 62 to 78 out of 96 episodes.
- Scoring Regime (Metaworld): On 168 new tasks, the "Average" method solved 127 tasks, whereas letting the model choose among the same windows solved at most 87.
- Diagnostic Robustness: The authors developed a heuristic rule to choose between "warm start" and "average" based on the failure regime. This rule correctly adapted even when the model was artificially degraded (quantized), solving 678 of 960 new PushT episodes compared to 530 for the best fixed strategy.
Limitations and Future Work
- Requirement for Demonstrations: The diagnostic and the remedy both rely on having access to a library of successful demonstrations.
- Short Horizons: The study focuses on short-horizon "window tasks." Extending this diagnosis to long-horizon planning where errors compound is a major next step.
- Simulation vs. Reality: While RoboCasa uses real-robot data, all evaluations were performed in simulation. Testing the "Average" remedy on physical hardware where safety and precision are paramount remains future work.
Summary updated