OSWAM: An Effective World Action Model with One-Step Action Generation

Guowen Zhang, Liyi Chen, Xu Hu, Pengfei Wang, Jingfan Chen, Chenhang He, Lei ZhangarXiv 2026 · arXiv

TL;DR

OSWAM (One-Step World Action Model) enables high-quality action generation for autonomous driving and robotics in just a single diffusion step. By replacing unstructured random noise with "imagination tokens" and using privileged distillation from future data, it achieves a 3× speedup over state-of-the-art models while maintaining competitive accuracy and real-time performance (40ms latency).

What Problem Does This Paper Address?

World Action Models (WAMs) are the current frontier for "Physical AI." Unlike standard models that just map images to actions, WAMs learn the underlying physics of the world by predicting future visual states. However, most high-performing WAMs rely on diffusion models, which typically require tens of iterative denoising steps to generate a coherent action sequence.

This creates a massive bottleneck for real-time applications like autonomous driving or high-frequency robot control. If you simply try to reduce the number of steps (e.g., to 1 or 2), the performance "collapses" because the model can't recover a complex, physically consistent trajectory from pure random noise in one go. As shown in the paper's analysis, dropping from 32 steps to 1 step can result in a 40% performance loss in robotics tasks.

Figure 2. Quantitative comparison of the Fast-WAM [4] test-time scaling with different action denoising steps on (a) the autonomous driving dataset Navsim, and (b) embodied control dataset RoboTwin.

Methods

The authors introduce OSWAM, which tackles the "one-step" challenge through three primary innovations:

  1. Imagination Tokens (AIT & SIT): Instead of starting the diffusion process with unstructured Gaussian noise, OSWAM uses a small set of learnable Action Imagination Tokens (AIT) and Scene Imagination Tokens (SIT). These tokens pass through a Video Diffusion Transformer (DiT) first, where they interact with the current visual frame to gain a "structured" physical prior before the action generation even begins.
  2. Privileged-Information Distillation: To make the model "future-aware" without actually generating slow video rollouts at test time, the authors use a teacher-student setup. During training, a teacher model sees the actual future video and distills that knowledge into the student’s imagination tokens. At inference, the student can use these "informed" tokens to act as if it has seen the future, using only the current frame.
  3. Role-aware 3D RoPE: The model uses a novel positional encoding scheme (Rotary Positional Embedding) that places video pixels, action sequences, and scene context into a shared 3D coordinate system, allowing the transformer to distinguish between different "roles" while maintaining temporal alignment.
The OSWAM architecture uses Video DiT to produce structured priors for the Action DiT.

Datasets

The model was evaluated across a diverse range of benchmarks:

  • NAVSIM: A modern autonomous driving simulation focused on planning metrics.
  • RoboTwin 2.0: A challenging bimanual (two-armed) robot manipulation benchmark with 50+ tasks.
  • nuScenes: A standard real-world autonomous driving dataset.
  • PiPER: Real-world evaluation using a physical robotic arm for tabletop manipulation tasks.

Key Insights and Ideas

  • Structure over Noise: The core "aha moment" is that the failure of one-step diffusion isn't necessarily a lack of model capacity, but a poor starting point. By providing a "structured initialization" via imagination tokens, the model doesn't have to "invent" physics from scratch in every frame; it just has to refine a well-informed guess.
  • Future-Awareness is Distillable: You don't need to see the future at test time if your latent representations have been trained to anticipate it. Distilling future-informed features into the current-frame tokens provides the benefits of a world model without the computational tax of video generation.
  • Efficiency is the Enabler: By reaching 40ms latency (on an RTX 5090), OSWAM moves WAMs from "interesting research" to "deployable technology" for 25Hz closed-loop control.

Key Results

OSWAM achieves a "Pareto-optimal" balance between speed and performance, often matching or exceeding models that take 10× more steps.

  • Speedup: It provides a ~3× speedup over direct-policy WAM baselines (like SimWAM and Fast-WAM).
  • Autonomous Driving: On NAVSIM, OSWAM achieved a PDMS score of 90.3, matching the 10-step SimWAM while being significantly faster.
  • Robotics: On RoboTwin 2.0, it reached a 90.0% average success rate, outperforming several multi-step models and few-step distillation methods.
  • Real-World Deployment: The model was successfully tested on a PiPER robot arm, completing tasks like "bread into bowl" with higher success rates and faster completion times than previous state-of-the-art methods.
Figure 1. NAVSIM and Robotwin 2.0 inference efficiency-performance comparison. EPT denotes embodied pretraining. All latencies are measured on NVIDIA H20 GPU.

Limitations and Future Work

While OSWAM significantly improves the initialization of the action generation process, the authors note that the Action DiT (the part that refines the tokens into final actions) doesn't yet receive direct privileged supervision. Future work could involve extending the distillation process to the action expert itself. Additionally, while the Scene Imagination Tokens (SIT) help, their potential to directly condition the action expert remains an unexplored avenue for even more robust control.

Summary updated

Abstract

Direct-policy World Action Models (WAM) have demonstrated the benefits of decoupling action generation from explicit future visual prediction for efficient inference in autonomous driving and embodied control. However, they still require tens of denoising steps to achieve competitive action quality, incurring substantial latency and hindering real-time deployment. Simply reducing the number of denoising steps will collapse the nonlinear multi-step denoising trajectory into coarse updates, substantially degrading action quality. To reduce the denoising steps while enabling real-time action generation, we present an effective world action model with one-step action generation, namely OSWAM, for autonomous driving and embodied control. We argue that it is the absence of structured action priors in random noise initialization that forces direct-policy WAMs to recover coherent action sequences through iterative denoising. Hence, we construct imagination tokens to initialize the diffusion process, providing physical priors and action-conditioned representations. We further introduce privileged information distillation with future conditioning to enhance future-aware action generation. As a result, OSWAM can efficiently and effectively generate high quality actions in just one diffusion step. Our experiments demonstrate that OSWAM achieves comparable or even better results in both autonomous driving and embodied control than direct-policy WAM and Joint-WAM that require tens of steps. It obtains approximately $3\times$ speedup over direct-policy WAM baselines and supports real-time compiled inference with latency as low as 40ms on an NVIDIA RTX 5090 GPU. The project page is available at https://gwenzhang.github.io/OSWAM/
BibTeX
@misc{zhang2026oswam,
  author = {Guowen Zhang and Liyi Chen and Xu Hu and Pengfei Wang and Jingfan Chen and Chenhang He and Lei Zhang},
  title = {{OSWAM: An Effective World Action Model with One-Step Action Generation}},
  year = {2026},
  eprint = {2610.35850},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.35850},
}
Figure