TL;DR
$\omega$-EVA introduces an "Envision–Verify–Act" loop for robot manipulation, allowing a policy to inspect the imagined consequences of its own action proposals before executing them. By performing this counterfactual reasoning entirely within a compact latent feature space, the model avoids slow pixel-level video generation, delivering a fast, robust, and highly competitive 1.2B-parameter policy without requiring large-scale robot pretraining.
What Problem Does This Paper Address?
Most modern vision-language-action (VLA) models follow a direct observation-to-action paradigm: they look at the current scene and immediately output a motor command. While effective, this approach leaves the future consequences of an action implicit within the network's weights. If a robot makes a slight error in alignment or grasp calculation, it has no way to "catch" its mistake before the physical hardware moves.
While world models can predict the future, existing robotics frameworks typically use them in one of two passive ways: either as a training-time auxiliary loss that is discarded during inference, or as a heavy, test-time video simulator that is far too slow for real-time, closed-loop control. $\omega$-EVA bridges this gap by turning the world model into an active, high-speed verification module directly embedded within the action-generation loop.
Methods
$\omega$-EVA implements its Envision–Verify–Act paradigm through a decoupled, three-stage training framework:
- Stage 1: Action-Conditioned Latent World Model ($W_\theta$): The model learns to predict future visual features (specifically from a frozen DINOv3 encoder) given the current observation and a variable-horizon action prefix. Crucially, a causal attention mask ensures that the current-state representation ($c_t$) remains independent of the action, yet highly sensitive to motion-relevant visual cues like object boundaries and contact points.
- Stage 2: World-Aware Action Generator ($\pi_\phi$): A language-conditioned policy is trained on top of the dynamics-aware representations from Stage 1. It uses conditional flow matching to generate an initial action chunk proposal ($a^0_{t:t+H}$).
- Stage 3: Imagined-Interaction Action Refiner ($R_\psi$): The world model and proposal policy are frozen. The initial action proposal is fed back into the world model to generate an imagined future latent ($\hat{I}_f$). Finally, a tri-branch refiner transformer jointly reasons over the current state, the imagined future, and the proposed action to output a corrected, final action chunk.
Because the entire "imagination" process happens in the compact latent space of DINOv3 features rather than decoding full RGB videos, the entire loop is fast enough for real-time deployment.
Datasets
The authors evaluated $\omega$-EVA across three major simulated manipulation benchmarks:
- LIBERO: Evaluates spatial reasoning, object interaction, goal interpretation, and long-horizon tasks across four distinct suites.
- LIBERO-PLUS: Tests policy robustness against seven controlled environmental perturbations (e.g., changes in lighting, camera angles, layout shifts, and sensor noise).
- RoboTwin 2.0: A bimanual manipulation benchmark featuring high action dimensionality, multiple camera views, and domain-randomized conditions.
Key Insights and Ideas
- Latent Space is All You Need for Imagination: By completely bypassing pixel-level video generation at inference time, $\omega$-EVA proves that a policy can reason about physical consequences using abstract feature dynamics alone. This keeps the control loop practical for real-time robotics (~40 Hz).
- The Synergy of Feedback and Anchoring: Through ablation studies, the authors discovered that the refiner needs both the imagined future and the original proposal. Removing the proposal branch (leaving only the current state and future prediction) caused a massive performance drop. The original proposal acts as a vital "correction anchor" for the refiner.
- Coarser Proposals are Cheaper to Fix: Surprisingly, the refiner doesn't need a perfectly denoised initial proposal. Reducing the flow-matching integration steps from 5 down to 1 step cut inference time from 45 ms to 25 ms while maintaining a near-identical success rate (98.8%). The refiner easily corrects the coarser, single-step proposal.
Key Results
Despite having only 1.2 billion parameters and using zero additional robot-data pretraining, $\omega$-EVA achieved state-of-the-art or highly competitive results:
- LIBERO: Reached a 98.6% average success rate, outperforming several much larger models (up to 8B parameters) that were pretrained on massive external robotics datasets.
- LIBERO-PLUS: Demonstrated strong robustness, boosting the average success rate from 81.2% (Stage 2 proposal) to 83.4% after Stage 3 refinement, showing particular strength against layout shifts and camera perturbations.
- RoboTwin 2.0: Successfully scaled the framework to bimanual control, improving the baseline average from 88.9% to 90.3% across clean and randomized environments.
Limitations and Future Work
While $\omega$-EVA demonstrates the power of latent verification, the authors note several areas for improvement:
- Single-Step Refinement: The current model only performs one envision-verify step per action chunk. Implementing an iterative imagination-refinement loop could allow deeper reasoning for complex tasks.
- Intra-Chunk Closed-Loop Control: The model currently replans after executing an action chunk. Continuous refinement during chunk execution using intermediate observations would make the robot more responsive.
- Beyond Vision: The predictive world model is currently purely visual. Incorporating tactile, force, and proprioceptive feedback into the latent imagination loop would significantly benefit contact-rich dexterous manipulation.
Summary updated