TL;DR
Episodic Kernel Repulsion (EKR) is a training-free, "plug-and-play" cost term that helps latent world models escape planning dead ends. By maintaining a small memory of visited states and repelling candidate plans away from them, EKR breaks the memoryless loops that cause agents to get stuck in "latent traps," significantly boosting success rates across navigation and manipulation tasks with minimal compute overhead.
What Problem Does This Paper Address?
Modern "zero-shot" visual control often relies on latent world models. These models plan by searching for action sequences that minimize the Euclidean distance between a predicted future state and a goal state in a learned latent space.
The fundamental issue is that latent distance is not geodesic distance. In environments with obstacles, the straight-line path in latent space might lead directly into a wall or a corner. Because standard Model Predictive Control (MPC) is "memoryless"—it only cares about the current state and the goal—the agent can fall into a latent trap. In these regions, every short-term plan that moves toward the goal looks worse (higher cost) than staying put or moving back into the trap. This creates a closed-loop attractor where the agent endlessly revisits the same unsuccessful states.
Methods
The authors propose Episodic Kernel Repulsion (EKR), a simple but effective modification to the planning cost function:
- Episodic Memory: The agent stores the latent representations of states it has actually visited during the current episode.
- Novelty Gating: To keep the memory efficient and spatially representative, a new state is only added if it is sufficiently far from existing entries, determined by an adaptive threshold based on recent trajectory spacing.
- Kernel Repulsion: During planning, EKR adds a penalty to candidate plans based on their proximity to the $k$ nearest neighbors in memory. This is calculated using a Gaussian kernel: $$\rho_K(\hat{z}^{(i)}) = \frac{1}{K_{eff}} \sum_{k=1}^{K_{eff}} \exp\left(-\frac{\|\hat{z}^{(i)} - m_{(k)}\|_2^2}{2\sigma^2}\right)$$
- Scale-Free Cost Fusion: To ensure the repulsion works across different models and latent dimensions without manual tuning, the penalty is rescaled by the mean "base cost" of the current batch of candidate plans. This makes the repulsion strength $\lambda$ a dimensionless hyperparameter.
Crucially, EKR is training-free. It requires no new neural networks and can be implemented in just a few lines of code around an existing cost function.
Datasets and Benchmarks
The method was evaluated using three different world model backbones (LeWM, H-JEPA, and DINO-WM) across several challenging benchmarks:
- Visual AntMaze: Navigation in a maze with a quadruped robot.
- FourRoom: A classic navigation task with narrow doorways.
- Push-T: A manipulation task requiring the agent to push a T-shaped object to a target.
- OGBench Cube: A short-horizon manipulation task.
- PointMaze: Long-horizon navigation tasks.
Key Insights and Ideas
- Traps are Plateaus: Inside a latent trap, the cost function isn't just pointing the wrong way; it's often flat. This means the model provides no signal on how to escape.
- History as a Heuristic: The agent already possesses the information needed to escape a trap: its own history. If it keeps returning to the same spot, that spot should become "expensive."
- Efficiency over Brute Force: Simply increasing the search budget (more samples or iterations) is often ineffective inside a trap because the planner just converges more precisely to the local minimum. EKR changes the landscape itself to remove the minimum.
- Transferability: Because of the scale-free cost fusion, a single configuration of EKR hyperparameters can transfer across entirely different model architectures (e.g., from a gradient-based JEPA model to a sampling-based DINO-WM) without re-tuning.
Key Results
EKR consistently outperformed baselines across all tested environments:
- Success Rates: On average, EKR raised success from 31.8% to 46.7% for LeWM and from 54.7% to 64.0% for the hierarchical H-JEPA model.
- Compute Efficiency: In 7 out of 8 settings, EKR with standard compute matched or exceeded a baseline using 4× more samples. In some cases, it even beat an 8× sample budget while using only ~1.5× the compute.
- Long-Horizon Gains: On a long-horizon PointMaze task, EKR improved success from 44.7% to 100.0%.
- Reduced Failure: Analysis showed a significant drop in "stuck" episodes and state revisits, confirming that the agent successfully navigates away from previously visited regions.
Limitations and Future Work
- Slow Escape from Dead Ends: EKR fills the local cost basin one memory entry at a time. In very deep dead ends, it may still take many steps to "fill" the trap enough to push the agent out.
- Repulsion Near the Goal: Since the memory doesn't know where the goal is, it can theoretically repel the agent away from a goal it has already visited or passed near. The authors suggest annealing the repulsion strength $\lambda$ as the base cost becomes small as a potential fix.
- Gradient Overhead: For gradient-based planners, differentiating through the repulsion term adds a small amount of wall-clock time per step (though peak memory remains unchanged).
Summary updated