TL;DR
GEMS (Generation and Experience Memory for Subgoals) is a new framework that significantly improves long-horizon planning for visual world models. By combining a time-aware subgoal generator with a "distance-gated" experience memory, GEMS provides precise, physically grounded targets that help robots navigate complex, multi-stage tasks where standard planning methods typically fail.
What Problem Does This Paper Address?
Visual world models allow robots to "imagine" the consequences of their actions, making them powerful tools for short-term planning. However, they struggle with "long-horizon" tasks—tasks that require many steps to complete, like picking up a cube and moving it to a distant location.
Two main issues arise:
- Compounding Error: Small inaccuracies in the world model's predictions accumulate over time, leading to "hallucinations" during long rollouts.
- Vanishing Guidance: In a high-dimensional latent space, the current state might look equally "dissimilar" to a distant goal regardless of which direction the robot moves, providing no useful signal for action selection.
Existing solutions use subgoals—intermediate targets—to break the task down. These are usually either generated (flexible but prone to physical inaccuracies) or retrieved from memory (grounded in reality but limited by what the robot has seen before). GEMS aims to combine the best of both worlds.
Methods
GEMS introduces a three-part architecture to guide a frozen world model planner:
- Temporally Structured Subgoal Generation: The system uses a "two-clock" approach. It separately encodes the time remaining to the final goal and the time to the next subgoal. Instead of predicting a subgoal from scratch, it predicts a residual (a correction) relative to an anchor point that is interpolated between the current state and the final goal based on progress.
- Latent Experience Memory (LEM): This is a non-parametric bank of encoded observations from training trajectories. It serves as a library of "real" states the robot can aim for.
- Selective Memory Grounding (The Distance Gate): This is the "secret sauce." The generated subgoal acts as a query to the memory. If a stored state is found within a certain distance threshold ($\lambda$), the planner targets that real, grounded state. If the memory is sparse and no close match exists, the system falls back to the generated subgoal.
The entire planning process uses the Cross-Entropy Method (CEM) to search for action sequences that bring the world model's predicted outcome closest to the selected subgoal.
Datasets
The authors evaluated GEMS across six diverse environments:
- OGBench: Cube and Cube-double (complex manipulation).
- Push-T: A standard benchmark for non-prehensile manipulation.
- TwoRoom and Reacher: Navigation and basic reaching tasks.
- Meta-World: A suite of six different manipulation tasks (e.g., opening a drawer, pressing a button).
Key Insights and Ideas
- Complementarity is Key: Generation provides the "intent" (where we should be in 15 steps), while memory provides the "precision" (exactly how the gripper should look when touching the object).
- The "Hard" Parts Benefit Most: Analysis showed that GEMS provides the biggest boost during the most difficult phases of a task, such as the initial grasp. Without memory grounding, generated subgoals often "skip ahead," imagining the cube is already lifted when it hasn't been grasped yet.
- Robustness to Horizon: While baseline methods see their success rates plummet as the goal gets further away, GEMS maintains high performance, showing it effectively decomposes long tasks into manageable chunks.
Key Results
GEMS set new state-of-the-art benchmarks across all manipulation tasks:
- Cube: Achieved 92.8% success, outperforming the strongest baseline (SAGE) by 24.0 percentage points.
- Push-T: Reached 68.4% success (a 7.3 point improvement).
- Meta-World: Reached 54.1% success (a 22.2 point improvement).
- Cube-double: While other methods failed completely (near 0%), GEMS achieved 17.7% success, demonstrating its ability to handle extremely sparse memory and complex multi-object coordination.
Limitations and Future Work
- Frozen World Model: GEMS relies on a pre-trained, frozen world model. If the underlying world model has fundamental blind spots, GEMS cannot overcome them.
- Data Dependency: The quality of the Experience Memory is tied to the training data. In entirely novel environments where the robot has zero prior experience, the system relies solely on the generator.
- Threshold Tuning: The distance gate threshold ($\lambda$) currently needs to be tuned per task using validation data. Automating this selection or making it dynamic could be a fruitful area for future research.
Summary updated