TL;DR
OpenWAM introduces an open, modular research stack designed to systematically deconstruct and optimize World-Action Models (WAMs) for robotic control. By decoupling the WAM design space into composable modules, the authors isolate the core principles driving world-action synergy and scale them into OpenWAM-$\alpha$, a foundation model pretrained on 6,400 hours of video and robot data. The resulting model achieves top-tier performance across eight simulation benchmarks and generalizes robustly to real-world robotic platforms.
What Problem Does This Paper Address?
World-Action Models (WAMs) represent a promising paradigm in embodied AI. By initializing from pretrained video-generation models, WAMs inherit rich visual and physical dynamics priors ("world knowledge") and attempt to channel them into executable robotic actions.
However, existing WAMs are monolithic and tightly coupled. Their generative backbones, visual representations, attention mechanisms, and training data are bound together in single implementations. This lack of modularity makes it impossible to conduct controlled experiments or determine why certain design choices succeed while others fail. OpenWAM addresses this gap by providing a factorized infrastructure to systematically study and optimize the interactions between world prediction and action generation.
Methods
The authors break down their contribution into three pillars:
- OpenWAM-Infra: A modular infrastructure that factorizes WAMs into three interchangeable classes of modules: visual encoders, stream backbones (video, action, and understanding streams), and visibility attention masks. It unifies training under a joint flow-matching objective and standardizes deployment with configurable inference modes and denoising schedules.
- OpenWAM-Study: A systematic empirical evaluation built on top of the infrastructure. It answers critical design questions regarding backbone scaling, visual representation choices, training-time information routing, and test-time denoising schedules.
- OpenWAM-$\alpha$: A scaled foundation model instantiated from the optimal recipe discovered in the study. It features a dual-system architecture combining a 5B parameter video backbone (Wan2.2-TI2V-5B) and a dedicated 1B parameter ActionDiT, bound by a mutual visibility attention mask.
Datasets
- Pretraining Data: A diverse mixture totaling 518.5M frames (~6,369 hours). This includes 30% egocentric human video (providing broad visual and interaction coverage) and 70% robot data split between real-world trajectories (AgiBotWorld-Beta, RoboCOIN, DROID) and synthetic simulation data (InternData-A1).
- Simulation Benchmarks: OpenWAM-Infra integrates eight distinct simulation environments for evaluation: LIBERO, LIBERO-Plus, VLABench, RoboTwin2.0, RoboDojo, RoboCasa365, EBench, and RoboCasa-GR1.
- Real-World Evaluation: Validated physically across single-arm manipulation (Franka-Research-3), bimanual tracking (RoboDojo platforms), and dexterous hand control (Wuji-hand + Tianji-arm).
Key Insights and Ideas
- Compact Latents Trump Encoder Type: The debate between reconstructive (VAE) and semantic representation (DINOv3/V-JEPA) encoders is less important than the operational properties of the latent space. Representation encoders perform on par with or better than reconstructive ones, provided their high-dimensional features are compressed into a compact, information-rich latent space via an adapter like an S-VAE.
- Bidirectional Synergy Emerges at Scale: When training from scratch on limited data, one-way information flow (actions reading video features) is sufficient. However, at a pretrained scale, a mutual visibility mask—where video generation and action prediction explicitly condition on each other bidirectionally—becomes consistently superior.
- Synchronized Denoising is Optimal: Testing various test-time denoising schedules revealed that forcing one modality to lead or lag the other degrades performance. Advancing both video and action streams in lockstep along the noise-plane diagonal yields the best results.
- Pretraining Drives OOD Generalization: Embodied pretraining provides modest gains for in-domain tasks but massively expands out-of-domain (OOD) generalization. One-stage co-training of human egocentric video and robot trajectories provides the best practical integration strategy.
Key Results
- State-of-the-Art on EBench: OpenWAM-$\alpha$ set a new state-of-the-art on the mobile bimanual benchmark EBench, outperforming the runner-up Qwen-RobotManip by roughly 4 points in both success rate and progress score.
- Top-Tier Simulation Performance: Across the remaining benchmarks, OpenWAM-$\alpha$ consistently placed in the top tier, outperforming other WAMs and matching or closely trailing the best Vision-Language-Action (VLA) models.
- Real-Robot Generalization: Achieved an 82.5% average success rate across six physical single-arm tasks. Crucially, when fine-tuned on dexterous hand tasks, it successfully generalized to a 21-DoF Wuji dexterous hand—an embodiment entirely absent from its pretraining mixture.
- Inference Efficiency: By leveraging DiT velocity caching and
torch.compile, the 10-step joint denoising loop executes in ~170 ms per chunk on an RTX 5090, comfortably meeting real-time control constraints.
Limitations and Future Work
- Post-Training Exploration: The current study focuses heavily on the pretraining phase; optimal strategies for downstream fine-tuning and task-specific adaptation remain open questions.
- Modality Fusion Architecture: The architecture currently relies on cross-attention or hard-routed Mixture-of-Experts (MoE). The authors suggest exploring early tokenization-phase fusion in future iterations.
- Pixel-Level Vulnerabilities: WAMs predicting future states in pixel latent spaces remain sensitive to extreme visual disturbances (e.g., camera shifts or lighting changes), indicating a need for more robust semantic-generative world models.
Summary updated