$ω$-EVA: Envision, Verify, and Act with Latent Interactive World Models

Zhenguo Sun, Yu Sun, Hande Huang, Alois KnollarXiv 2026 · arXiv

TL;DR

$\omega$-EVA introduces an "Envision–Verify–Act" loop for robot manipulation, allowing a policy to inspect the imagined consequences of its own action proposals before executing them. By performing this counterfactual reasoning entirely within a compact latent feature space, the model avoids slow pixel-level video generation, delivering a fast, robust, and highly competitive 1.2B-parameter policy without requiring large-scale robot pretraining.

What Problem Does This Paper Address?

Most modern vision-language-action (VLA) models follow a direct observation-to-action paradigm: they look at the current scene and immediately output a motor command. While effective, this approach leaves the future consequences of an action implicit within the network's weights. If a robot makes a slight error in alignment or grasp calculation, it has no way to "catch" its mistake before the physical hardware moves.

While world models can predict the future, existing robotics frameworks typically use them in one of two passive ways: either as a training-time auxiliary loss that is discarded during inference, or as a heavy, test-time video simulator that is far too slow for real-time, closed-loop control. $\omega$-EVA bridges this gap by turning the world model into an active, high-speed verification module directly embedded within the action-generation loop.

Methods

$\omega$-EVA implements its Envision–Verify–Act paradigm through a decoupled, three-stage training framework:

Figure 1: Overview of ω-EVA. The left side shows the Envision–Verify–Act interaction loop: the world-aware action generator proposes an action, the action-conditioned latent world model envi- sions its consequence, and the imagined-interaction refiner corrects the proposal before execution. The right side details the three modules learned across Stages 1–3.
  1. Stage 1: Action-Conditioned Latent World Model ($W_\theta$): The model learns to predict future visual features (specifically from a frozen DINOv3 encoder) given the current observation and a variable-horizon action prefix. Crucially, a causal attention mask ensures that the current-state representation ($c_t$) remains independent of the action, yet highly sensitive to motion-relevant visual cues like object boundaries and contact points.
  2. Stage 2: World-Aware Action Generator ($\pi_\phi$): A language-conditioned policy is trained on top of the dynamics-aware representations from Stage 1. It uses conditional flow matching to generate an initial action chunk proposal ($a^0_{t:t+H}$).
  3. Stage 3: Imagined-Interaction Action Refiner ($R_\psi$): The world model and proposal policy are frozen. The initial action proposal is fed back into the world model to generate an imagined future latent ($\hat{I}_f$). Finally, a tri-branch refiner transformer jointly reasons over the current state, the imagined future, and the proposed action to output a corrected, final action chunk.
Figure 2: Stage 2 world-aware action generation. A time-conditioned Query Transformer takes the noised action chunk as query tokens. Each block performs action self-attention followed by cross-attention to the world-model representation ct and language tokens Te, and predicts the flow velocity used to generate an initial action proposal.

Because the entire "imagination" process happens in the compact latent space of DINOv3 features rather than decoding full RGB videos, the entire loop is fast enough for real-time deployment.

Datasets

The authors evaluated $\omega$-EVA across three major simulated manipulation benchmarks:

  • LIBERO: Evaluates spatial reasoning, object interaction, goal interpretation, and long-horizon tasks across four distinct suites.
  • LIBERO-PLUS: Tests policy robustness against seven controlled environmental perturbations (e.g., changes in lighting, camera angles, layout shifts, and sensor noise).
  • RoboTwin 2.0: A bimanual manipulation benchmark featuring high action dimensionality, multiple camera views, and domain-randomized conditions.

Key Insights and Ideas

  • Latent Space is All You Need for Imagination: By completely bypassing pixel-level video generation at inference time, $\omega$-EVA proves that a policy can reason about physical consequences using abstract feature dynamics alone. This keeps the control loop practical for real-time robotics (~40 Hz).
  • The Synergy of Feedback and Anchoring: Through ablation studies, the authors discovered that the refiner needs both the imagined future and the original proposal. Removing the proposal branch (leaving only the current state and future prediction) caused a massive performance drop. The original proposal acts as a vital "correction anchor" for the refiner.
  • Coarser Proposals are Cheaper to Fix: Surprisingly, the refiner doesn't need a perfectly denoised initial proposal. Reducing the flow-matching integration steps from 5 down to 1 step cut inference time from 45 ms to 25 ms while maintaining a near-identical success rate (98.8%). The refiner easily corrects the coarser, single-step proposal.

Key Results

Despite having only 1.2 billion parameters and using zero additional robot-data pretraining, $\omega$-EVA achieved state-of-the-art or highly competitive results:

  • LIBERO: Reached a 98.6% average success rate, outperforming several much larger models (up to 8B parameters) that were pretrained on massive external robotics datasets.
  • LIBERO-PLUS: Demonstrated strong robustness, boosting the average success rate from 81.2% (Stage 2 proposal) to 83.4% after Stage 3 refinement, showing particular strength against layout shifts and camera perturbations.
  • RoboTwin 2.0: Successfully scaled the framework to bimanual control, improving the baseline average from 88.9% to 90.3% across clean and randomized environments.

Limitations and Future Work

While $\omega$-EVA demonstrates the power of latent verification, the authors note several areas for improvement:

  • Single-Step Refinement: The current model only performs one envision-verify step per action chunk. Implementing an iterative imagination-refinement loop could allow deeper reasoning for complex tasks.
  • Intra-Chunk Closed-Loop Control: The model currently replans after executing an action chunk. Continuous refinement during chunk execution using intermediate observations would make the robot more responsive.
  • Beyond Vision: The predictive world model is currently purely visual. Incorporating tactile, force, and proprioceptive feedback into the latent imagination loop would significantly benefit contact-rich dexterous manipulation.

Summary updated

Abstract

Embodied policies typically map current observations directly to actions, leaving candidate-action consequences implicit. World models provide predictive supervision, representations, or external simulation, but rarely let a policy inspect the imagined consequence of its own proposal before acting. We introduce $ω$-EVA, a latent interactive world model that realizes an Envision--Verify--Act loop for embodied action generation. Its three-stage framework learns action-conditioned latent dynamics, trains a language-conditioned flow policy on dynamics-aware visual representations, and feeds the policy's proposal back through the world model. A tri-branch refiner jointly reasons over the current state, proposal-conditioned future, and proposed action to produce the final action chunk. Because consequence reasoning remains in latent feature space, $ω$-EVA avoids generating future videos at inference. Evaluations across diverse single-arm, bimanual, long-horizon, and perturbed simulation settings show that the complete interaction pipeline consistently improves the proposal policy, while latent diagnostics indicate meaningful action-conditioned future structure. With approximately 1.2B parameters and no additional robot-data pretraining, $ω$-EVA demonstrates a compact and competitive performance--scale--data trade-off, making the world model an active action-feedback module rather than a passive predictor.
BibTeX
@misc{sun2026eva,
  author = {Zhenguo Sun and Yu Sun and Hande Huang and Alois Knoll},
  title = {{$ω$-EVA: Envision, Verify, and Act with Latent Interactive World Models}},
  year = {2026},
  eprint = {2606.09457},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2606.09457},
}

Loading related content...

Figure