When More Planning Hurts: Learned Planners for World Models

Josef ChenarXiv 2026 · arXiv

In the world of robot learning, the prevailing wisdom is often "more is better." More data, more parameters, and certainly more planning. If your robot isn't solving a task, the intuitive fix is to give it more "thinking time"—more optimization steps to find a better plan.

A new paper by Josef Chen, "When More Planning Hurts: Learned Planners for World Models," challenges this fundamental assumption. Through a series of rigorous experiments on a visual manipulation task, Chen demonstrates a counterintuitive reality: in the presence of imperfect world models, extra planning effort can actually lead to worse performance.

TL;DR

This paper investigates the relationship between planning effort and task success in latent world models. Surprisingly, it finds that increasing the number of optimization updates can sometimes turn a successful robot trajectory into a failure, even when the planner successfully finds a "better" (lower-cost) path according to the model.

What Problem Does This Paper Address?

Model-based reinforcement learning relies on a "world model" to predict the outcomes of actions. A planner then searches this model to find a sequence of actions that minimizes a cost function. However, these models are never perfect.

The authors tackle the objective mismatch problem: the gap between what the model thinks is a good plan (low predicted cost) and what actually works in the real world (task success). Specifically, they ask: does spending more computational effort to optimize a plan within a learned world model always yield better results at execution time?

Methods

The study focuses on the PushT task, a simulated visual manipulation challenge where a robot must push a T-shaped block into a target pose.

The researchers compared several planning approaches:

  • Fixed Action Updater: A tiny, supervised neural network (only 1,505 parameters) designed to refine plans.
  • Conventional Optimizers: Standard methods like Adam and iCEM (improved Cross-Entropy Method).
  • LePlanner: A previously released, high-performance learned controller.

The core of the paper is a matched intervention study. Starting from the exact same state and history, the authors varied only the "planning budget"—the number of updates allowed (4, 8, or 12). This allowed them to isolate the effect of planning effort on the final outcome.

Datasets

  • PushT: The primary simulation environment for visual manipulation.
  • Reserved Scenes: A set of 16 specific starting configurations (scenes) used for the final "confirmation" evaluation to ensure the results weren't just a fluke of the training data.
  • Development Scenes: An initial set of 24 scenes used for tuning and preliminary experiments.

Key Insights and Ideas

  • The "More Planning Hurts" Phenomenon: In a matched experiment, increasing the planning budget from 4 to 12 updates actually decreased the success rate. Tasks that were solved with 4 updates became failures with 12.
  • Optimization Success vs. Execution Failure: The planners weren't "failing" in the traditional sense; they were successfully finding plans with lower predicted costs. However, these "better" plans led the robot into regions where the world model's inaccuracies caused the task to fail.
  • The Hidden Danger of Lower Scores: Figure 1 perfectly illustrates this. A plan with 12 updates achieved a much lower latent score (10.294 vs 19.676) but failed because it misjudged the required pusher position, leading to a massive joint error.
Figure 1. Similar block poses can hide a failed goal. In scene 18651, the official predictor gets a ten-command history and follows the plans with four and twelve updates. For the plan with four updates, the task succeeds at command 25. For the plan with twelve updates, the selected score is lower and the task fails at command 50. This is the only solve that differs between these two settings; in all other solves, eight updates were used. Success requires matching not only the block position and orientation, but also the pusher position. The recorded poses at the end of the episode are used to check the pusher’s distance from the goal and the joint position error. The two displayed angle errors are both within tolerance. In the figure, gold marks the goals in the dataset; blue and black show the recorded block and pusher poses, respectively; dotted lines indicate the pusher position error. All results in the matched experiment for every scene/model/effort combination are shown in the bottom panel. The scene in which there is a difference is highlighted.
  • Model Inaccuracy as a Ceiling: The quality of the world model acts as a hard ceiling. If the model is wrong about the physics of a certain interaction, no amount of planning "inside" that model will fix the resulting real-world error. In fact, aggressive optimization might just exploit the model's flaws more effectively.

Key Results

The quantitative results highlight the struggle of the learned updater against established baselines:

  • Success Rates (out of 16 scenes):
    • LePlanner: 16/16 (The gold standard)
    • Learned Updater (8 updates): 8/16
    • Adam: 7/16
    • iCEM: 2/16
Figure 2A. Task success rates across different controllers on the 16 reserved scenes.
  • The Planning Effort Trade-off: In the matched experiment, the success counts were 9/16 for 4 updates, 8/16 for 8 updates, and 7/16 for 12 updates. Extra effort rescued zero failing tasks but caused two previously successful ones to fail.
  • Efficiency: While the learned updater didn't beat LePlanner in success, it was significantly faster than Adam, using only about 27% of the planning and encoding time.

Limitations and Future Work

The author is transparent about several limitations:

  • Scope: The findings are based on a single task (PushT) and a relatively small number of scenes.
  • Architecture: The study doesn't fully isolate the effect of the controller architecture from other factors like training data and objectives.
  • Generalization: It's unclear if these results hold for different model families or real-world robotic hardware.

Future directions include testing these ideas on a wider variety of tasks, exploring real-robot deployments, and investigating whether "cheap" contextual policies (knowing when to stop planning) can mitigate the observed harms of over-optimization.

Summary updated

Abstract

On PushT, our fixed action updater (1,505 parameters) solves 8 of 16 scenes reserved from our configuration selection. The comparison planners perform as follows: Selected Adam (7 tasks solved, total planning and encoding time 218.440s), adapted iCEM (2 tasks solved, total planning and encoding time 49.309s), released LePlanner (16 tasks solved, total planning and encoding time 5.652s). The total planning and encoding time for our action updater is 59.079s. Note that our action updater does not satisfy the task-cost criterion we fixed before evaluation. We also conduct another matched experiment where we only replay the first ten commands in each context and then allow the next solve 4, 8 or 12 updates (subsequent solves all get 8 updates). Among all 16 contexts (8 scene clusters), the 3 budgets get 9, 8 and 7 successes, respectively. Interestingly, the extra updates did not rescue any failing task in this experiment. All the changes in success/failure happen in one scene cluster, where we observe that the extra updates decrease the costs of the selected latent world-model trajectories for our updater, while the tasks that used to be successes become failures. The code, weights, evaluators, execution traces and cost accounting accompany the paper.
BibTeX
@misc{chen2026when,
  author = {Josef Chen},
  title = {{When More Planning Hurts: Learned Planners for World Models}},
  year = {2026},
  eprint = {2610.37891},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.37891},
}
Figure