Learning Latent Goal Costs for Visual World-Model Planning with an Expanding Curriculum

Konstantinos Mylonas, Thrasyvoulos SpyropoulosarXiv 2026 · arXiv

TL;DR

This paper introduces a task-agnostic method to improve visual world-model planning by fitting a small "adapter" to frozen latent embeddings. By using an expanding curriculum of starting positions and leveraging the temporal order of successful trajectories as a training signal, the authors significantly improve planning success in obstacle-rich environments without requiring expert labels or retraining the underlying world model.

What Problem Does This Paper Address?

Visual latent world models allow agents to "imagine" the consequences of their actions in a compressed representation space. To reach a goal, a planner (like the Cross-Entropy Method, or CEM) rolls out candidate action sequences and scores the predicted outcomes against a goal image. Typically, this score is the Mean Squared Error (MSE) between the latent embeddings of the predicted state and the goal.

However, visual similarity does not always equal reachability. In an environment with obstacles, a state might be visually very similar to the goal (e.g., just on the other side of a thin wall) but require a long, complex path to reach. Standard pretrained encoders like I-JEPA or DINOv2 capture spatial features well but lack the "task geometry" needed to navigate these detours. As shown in Figure 4, raw latent costs often fail to reflect the true geodesic (shortest legal path) distance to a goal.

Raw latent-cost heatmaps compared to legal path distance. Darker colors indicate lower cost. Note how raw costs (DINOv2, I-JEPA) often favor states across walls that are actually far away in terms of travel.

Methods

The authors propose a system that keeps the pretrained visual encoder and the world model dynamics frozen, adding only a small residual adapter (a shared MLP) to the latent embeddings used for goal scoring.

The key innovation is the Expanding Curriculum:

  1. Band-based Collection: The agent starts by attempting very easy tasks—starting positions very close to the goal (Band 1). At this range, even raw latent cost is usually enough to guide the agent.
  2. Temporal Ranking: For every successful trajectory, the agent assumes that observations visited later are "better" (closer to the goal) than those visited earlier. This provides a weak, task-agnostic supervision signal.
  3. Adapter Fitting: The adapter is trained to minimize a ranking loss, ensuring that "later" observations have a lower adapted latent cost than "earlier" ones.
  4. Expansion: Once the adapter is trained on the current band, the starting region is expanded (e.g., Band 2, Band 3), and the process repeats. The improved adapter helps the agent succeed from further away, providing more data for the next stage of training.
The expanding curriculum bands in DotWall. Each stage adds starting positions further from the goal to collect increasingly difficult trajectories.

The method also incorporates data from failed rollouts by ranking them against successful routes using a nearest-neighbor approach in latent space.

Datasets

The researchers evaluated their approach on two simulated navigation environments:

  • DotWall: A ball must move between two rooms separated by a wall with a small doorway.
  • PointMaze: A U-shaped maze where the agent must often move away from the goal in Euclidean space to eventually reach it.
  • Generalization Test: A set of 16 unseen DotWall layouts where the wall and doorway positions were shifted to test if the adapter learned general navigation principles rather than just memorizing a specific map.

Key Insights and Ideas

  • Leveraging Pretrained Features: Instead of expensive end-to-end training, the method treats high-quality visual features as a foundation and only learns the "cost geometry" on top.
  • Success as Supervision: By using the temporal order of successful runs, the method avoids the need for "oracle" labels like ground-truth coordinates or geodesic distances, which are often unavailable in real-world robotics.
  • The "Aha" Moment for Detours: The curriculum is essential because it allows the agent to "bootstrap" its way out of local minima. By learning that the doorway is the path to the goal while nearby, the agent can eventually plan a path to that doorway from the other side of the room.

Key Results

The adapted latent cost led to dramatic improvements in planning success:

  • DotWall: Success rate jumped from 54% to 98%. Specifically, in cases where the direct path was blocked by a wall, success rose from 17% to 96%.
  • PointMaze: Success rate increased from 82% to 96%.
  • Generalization: On unseen DotWall layouts, the adapter improved the agreement between latent cost and the true shortest path from 71.9% to 85.8%. This suggests the adapter learned to prioritize features related to "path-finding" (like the doorway) rather than just visual proximity.
A matched example in DotWall. The raw-cost planner (left) gets stuck against the wall; the adapted-cost planner (right) successfully navigates through the doorway.

Limitations and Future Work

  • Curse of Dimensionality: The authors note that as the state space grows (e.g., more complex robot joints or objects), the number of starting positions required for the curriculum might increase exponentially.
  • Goal-Success Signal: The method still requires a binary signal of whether a goal was reached to filter training data.
  • Ranking Noise: Temporal order is an "imperfect assumption" because successful routes can include loops or detours. While the method was robust to this noise in simple mazes, more complex environments might require stricter data filtering.
  • Future Direction: The authors aim to deploy this method in higher-dimensional tasks, such as robotic manipulation.

Summary updated

Abstract

Image-goal planning with a frozen visual world model depends on how predicted observations are scored against the goal. Mean-squared error (MSE) between embeddings from a pretrained visual encoder can provide a useful signal near a goal, yet favor states across an obstacle that are visually similar to the goal and difficult to reach. We fit a small residual adapter on top of frozen I-JEPA embeddings to improve this latent goal cost. Starting near each goal, a planner collects trajectories from progressively wider regions of initial states. The adapter learns from the order of observations along successful trajectories, with additional comparisons inferred from failed rollouts. No task-specific progress labels are used for fitting, so the procedure can be applied to other goal-reaching tasks. DotWall and PointMaze are used in this work. In paired 50-case evaluations with identical starts, goals, world models, and planner settings, success increases from 54% to 98% in DotWall and from 82% to 96% in PointMaze. On 16 unseen DotWall layout-goal cases, the adapter fitted on one fixed layout improves agreement between latent-cost ordering and shortest legal-path ordering from 71.9% to 85.8%. The results indicate that planning experience can improve a frozen visual representation's goal cost without retraining the encoder or the world model.
BibTeX
@misc{mylonas2026learning,
  author = {Konstantinos Mylonas and Thrasyvoulos Spyropoulos},
  title = {{Learning Latent Goal Costs for Visual World-Model Planning with an Expanding Curriculum}},
  year = {2026},
  eprint = {2611.17529},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.17529},
}
Figure