GEMS: Generation and Experience Memory for Subgoal-Guided Long-Horizon World Model Planning

Yangchen Zhao, Ali Payani, Caiwen DingarXiv 2026 · arXiv

TL;DR

GEMS (Generation and Experience Memory for Subgoals) is a new framework that significantly improves long-horizon planning for visual world models. By combining a time-aware subgoal generator with a "distance-gated" experience memory, GEMS provides precise, physically grounded targets that help robots navigate complex, multi-stage tasks where standard planning methods typically fail.

What Problem Does This Paper Address?

Visual world models allow robots to "imagine" the consequences of their actions, making them powerful tools for short-term planning. However, they struggle with "long-horizon" tasks—tasks that require many steps to complete, like picking up a cube and moving it to a distant location.

Two main issues arise:

  1. Compounding Error: Small inaccuracies in the world model's predictions accumulate over time, leading to "hallucinations" during long rollouts.
  2. Vanishing Guidance: In a high-dimensional latent space, the current state might look equally "dissimilar" to a distant goal regardless of which direction the robot moves, providing no useful signal for action selection.

Existing solutions use subgoals—intermediate targets—to break the task down. These are usually either generated (flexible but prone to physical inaccuracies) or retrieved from memory (grounded in reality but limited by what the robot has seen before). GEMS aims to combine the best of both worlds.

Methods

GEMS introduces a three-part architecture to guide a frozen world model planner:

  1. Temporally Structured Subgoal Generation: The system uses a "two-clock" approach. It separately encodes the time remaining to the final goal and the time to the next subgoal. Instead of predicting a subgoal from scratch, it predicts a residual (a correction) relative to an anchor point that is interpolated between the current state and the final goal based on progress.
  2. Latent Experience Memory (LEM): This is a non-parametric bank of encoded observations from training trajectories. It serves as a library of "real" states the robot can aim for.
  3. Selective Memory Grounding (The Distance Gate): This is the "secret sauce." The generated subgoal acts as a query to the memory. If a stored state is found within a certain distance threshold ($\lambda$), the planner targets that real, grounded state. If the memory is sparse and no close match exists, the system falls back to the generated subgoal.
Figure 2. Overview of GEMS. The generator proposes a subgoal τ steps ahead, and the experience memory grounds it in a stored frame when one lies close. CEM with the frozen world model plans τ actions toward the resulting target; GEMS executes them and replans. Right: one Cube rollout, with the frames observed after each segment and the subgoals they aimed at. For visibility, frames are re-rendered from the recorded states with a lighter floor.

The entire planning process uses the Cross-Entropy Method (CEM) to search for action sequences that bring the world model's predicted outcome closest to the selected subgoal.

Datasets

The authors evaluated GEMS across six diverse environments:

  • OGBench: Cube and Cube-double (complex manipulation).
  • Push-T: A standard benchmark for non-prehensile manipulation.
  • TwoRoom and Reacher: Navigation and basic reaching tasks.
  • Meta-World: A suite of six different manipulation tasks (e.g., opening a drawer, pressing a button).

Key Insights and Ideas

  • Complementarity is Key: Generation provides the "intent" (where we should be in 15 steps), while memory provides the "precision" (exactly how the gripper should look when touching the object).
  • The "Hard" Parts Benefit Most: Analysis showed that GEMS provides the biggest boost during the most difficult phases of a task, such as the initial grasp. Without memory grounding, generated subgoals often "skip ahead," imagining the cube is already lifted when it hasn't been grasped yet.
  • Robustness to Horizon: While baseline methods see their success rates plummet as the goal gets further away, GEMS maintains high performance, showing it effectively decomposes long tasks into manageable chunks.
Figure 1. Long-horizon planning with experience-grounded subgoals. Left: planning toward the final goal alone never picks up the cube, whereas GEMS reaches the goal by planning toward intermediate subgoals. The subgoals (orange borders) are real frames selected from experience memory. Right: Cube success rates of GEMS and final-goal planning as the goal lies further ahead.

Key Results

GEMS set new state-of-the-art benchmarks across all manipulation tasks:

  • Cube: Achieved 92.8% success, outperforming the strongest baseline (SAGE) by 24.0 percentage points.
  • Push-T: Reached 68.4% success (a 7.3 point improvement).
  • Meta-World: Reached 54.1% success (a 22.2 point improvement).
  • Cube-double: While other methods failed completely (near 0%), GEMS achieved 17.7% success, demonstrating its ability to handle extremely sparse memory and complex multi-object coordination.
Figure 3. Planning across goal horizons on PushT. Each axis is a goal horizon H; a larger radius means higher success.

Limitations and Future Work

  • Frozen World Model: GEMS relies on a pre-trained, frozen world model. If the underlying world model has fundamental blind spots, GEMS cannot overcome them.
  • Data Dependency: The quality of the Experience Memory is tied to the training data. In entirely novel environments where the robot has zero prior experience, the system relies solely on the generator.
  • Threshold Tuning: The distance gate threshold ($\lambda$) currently needs to be tuned per task using validation data. Automating this selection or making it dynamic could be a fruitful area for future research.

Summary updated

Abstract

Visual world models enable effective short-horizon goal-conditioned planning, such as reaching nearby targets and pushing objects over short distances, by predicting candidate action outcomes and comparing their representations with a goal representation. However, extending this approach to long-horizon tasks is challenging: prediction errors accumulate over extended rollouts, while latent-space similarity to distant goals provides limited guidance for ranking candidate outcomes. Existing methods address these limitations by introducing intermediate subgoals, either generated by learned models or selected from experience memory. Generated subgoals can contain hallucinations or be inaccurate in aspects such as object positions and timing. Memory-based approaches are constrained by experience coverage and may lack precise guidance for selecting appropriate intermediate states from stored observations. We introduce GEMS, which generates temporally structured subgoals to guide action search and state selection from Latent Experience Memory (LEM). A distance gate determines whether predicted outcomes are evaluated against the generated subgoal or the selected memory state. Experiments across six tasks and task suites spanning manipulation, navigation, and reaching demonstrate the effectiveness of GEMS for long-horizon planning. GEMS outperforms state-of-the-art world-model planning baselines across different manipulation settings, achieving success rates of 92.8% on Cube, 68.4% on Push-T, and 54.1% on Meta-World. It improves over the strongest evaluated baseline on these tasks by 24.0, 7.3, and 22.2 percentage points, respectively. These results demonstrate that generated subgoals and experience memory can jointly strengthen long-horizon planning with a frozen world model.
BibTeX
@misc{zhao2026gems,
  author = {Yangchen Zhao and Ali Payani and Caiwen Ding},
  title = {{GEMS: Generation and Experience Memory for Subgoal-Guided Long-Horizon World Model Planning}},
  year = {2026},
  eprint = {2611.25134},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.25134},
}
Figure