What Makes World Action Models Generalize? An Empirical Study of Test-Time Future Modeling

Renping Zhou, Zanlin Ni, Zihao Fan, Guohao Fu, Zeyu Liu, Hao Shi, Jie Zhang, Chi Bene Chen, Yang Yue, Xueyang Fu, Gao HuangarXiv 2026 · arXiv

TL;DR

World Action Models (WAMs) gain their impressive generalization from predicting the future, but the iterative video denoising required at inference makes them painfully slow. This paper reveals a surprising "aha" moment: you don't actually need to generate clean future frames to get the benefits—a single forward pass over fully noised future tokens is sufficient. The authors introduce Simple-WAM, which matches the speed of fast "latent" models while significantly outperforming them in robustness and task generalization.

What Problem Does This Paper Address?

In embodied AI, we want robot policies that don't just memorize training data but generalize to new environments and tasks. World Action Models (WAMs) attempt this by jointly predicting future video frames and robot actions. By leveraging large-scale video pre-training, WAMs inherit rich spatiotemporal priors about how the world moves.

However, there is a major efficiency bottleneck. Explicit WAMs perform iterative denoising at test-time to generate clean future frames, which is computationally expensive and slow. To fix this, researchers proposed Latent WAMs, which predict the future during training but discard the future-modeling branch entirely at inference for speed. While Latent WAMs perform well on tasks they've seen before, this paper asks a critical question: Does discarding the future at test-time kill the very generalization that makes WAMs special?

Methods

The authors conducted a controlled empirical study comparing explicit and latent paradigms across three axes: environmental perturbation, data efficiency, and task generalization.

Their investigation led to two key findings:

  1. Generalization Gap: Latent WAMs match explicit ones in-distribution but fail significantly when the environment changes or data is scarce.
  2. The "Noisy" Shortcut: Most of the generalization benefit comes from the first denoising step. The action expert needs the representation of the future, not the final clean pixels.

Based on this, they proposed Simple-WAM:

  • Inference: Instead of 10+ denoising steps, it performs a single forward pass where future video tokens are kept as pure Gaussian noise ($\tau=1$). The action expert conditions on these noisy representations.
  • Training: They introduce a Mixed Schedule. During training, the model is occasionally forced to predict from pure noise (probability $p=0.5$). This "sharpens" the video expert's ability to extract useful features from noise, aligning training behavior with the simplified inference.
The three conditioning schemes: Explicit (slow), Latent (fast but weak), and Simple-WAM (fast and robust).

Datasets

The study utilizes several rigorous benchmarks:

  • LIBERO: A suite for spatial, object-centric, and long-horizon robot tasks.
  • LIBERO-Plus: A benchmark specifically designed to test robustness against seven types of environmental perturbations (e.g., lighting, background, camera angles).
  • RoboTwin 2.0: A bimanual manipulation benchmark.
  • Real-World Evaluation: Experiments conducted on an AgileX Aloha dual-arm platform across four tasks, including a held-out "Store in Order" task.

Key Insights and Ideas

  • Preparing vs. Generating: The paper argues that the benefit of WAMs comes from preparing the future. The video backbone provides a "template" of temporal evolution that the action expert uses to ground its motor commands.
  • In-Distribution is Deceptive: If you only look at success rates on training tasks, Latent WAMs look great. The "Generalization Deteriorates" finding (F1 in Figure 1) shows that future modeling is a test-time requirement for robustness, not just a training objective.
  • Noise as a Feature: Surprisingly, pure Gaussian noise tokens, when processed by a pre-trained video DiT (Diffusion Transformer), provide enough spatiotemporal context for the action expert to generalize. Replacing these with zeros or learnable embeddings causes performance to crater, proving that the video backbone's pre-trained knowledge is specifically tied to handling noised inputs.
Success rate vs. denoising steps. Almost all generalization gains are captured in the very first pass (t_denoise=1).

Key Results

  • Speed: Simple-WAM runs at 74.7 ms per action chunk, which is 3.8x faster than explicit WAMs (286.9 ms) and nearly as fast as latent models (62.0 ms).
  • Robustness: On LIBERO-Plus (environmental perturbations), Simple-WAM achieved a 79.5% success rate, crushing the latent paradigm (53.8%) and even beating the explicit paradigm (67.7%).
  • Task Generalization: When given action-free video of a new task, Simple-WAM's success rate jumped from ~2% to 73.6%, whereas the latent model remained stuck at ~6%.
  • Real-World: In physical robot tests, Simple-WAM maintained a 69.2% success rate under perturbations, while the latent model dropped to 25.2%.
Figure 1: An empirical study of test-time future modeling for WAM generalization. Left: the three axes of the protocol, environmental perturbation, data efficiency, and task generalization. Right: the explicit and latent paradigms are within 0.9 points on in-distribution tasks yet separate on all three axes (F1), and a single forward pass over a fully noised future recovers almost all of that separation (F2).

Limitations and Future Work

  • Backbone Specificity: The study primarily uses a 5B-parameter Wan2.2 video DiT. While the findings are robust, it's unclear if they hold for significantly smaller or differently architected backbones.
  • Scaling: The authors note that whether these conclusions hold at even larger scales (e.g., 10B+ parameters) or with massive embodied pre-training remains an open question.
  • Future Direction: Exploring how to further "distill" the spatiotemporal features from the video expert to reduce the 1.2x latency overhead compared to latent models.

Summary updated

Abstract

World action models (WAMs) predict the future alongside actions during training. Due to the heavy computation cost of video denoising, whether the future must still be generated during inference is disputed: Explicit WAMs denoise it into clean frames along with every action chunk, whereas Latent WAMs discard it entirely for acceleration. We find that latent WAMs, despite matching explicit ones on in-distribution tasks, fail to retain the generalization benefits that originally motivated WAMs. To demonstrate this, we evaluate generalization along three axes: environmental perturbation, data efficiency, and task generalization. Controlled comparisons with a matched backbone, training data, and budget reveal consistent degradation across all three axes when the action expert no longer conditions on future representations. Further analysis shows that the gap arises almost entirely from the first denoising step: the benefit comes from preparing the future, not generating it. We therefore propose Simple-WAM, which simplifies future modeling into a single forward pass of fully noised video tokens and adapts the training-time noise schedule to this inference behavior. Across simulation and real-world tasks, Simple-WAM achieves the best of both worlds, leading explicit WAMs in generalization performance with efficiency comparable to Latent WAMs. Project Page: https://zrporz.github.io/Simple-WAM-Web/
BibTeX
@misc{zhou2026what,
  author = {Renping Zhou and Zanlin Ni and Zihao Fan and Guohao Fu and Zeyu Liu and Hao Shi and Jie Zhang and Chi Bene Chen and Yang Yue and Xueyang Fu and Gao Huang},
  title = {{What Makes World Action Models Generalize? An Empirical Study of Test-Time Future Modeling}},
  year = {2026},
  eprint = {2609.34981},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2609.34981},
}
Figure