TL;DR
WAM-Policy enables arm-free quadrupedal robots to perform complex "loco-manipulation" tasks—like opening doors or kicking balls—using their front legs, all guided by natural language instructions. By adapting pre-trained arm-centric World Action Models (WAMs) through a two-stage process of on-policy distillation and reinforcement learning, the authors achieve high task success rates with a 14x inference speedup, making real-time control feasible.
What Problem Does This Paper Address?
Quadrupedal robots are prized for their ability to traverse rugged terrain where wheeled robots fail. However, to interact with the world, they typically require an expensive, heavy, and mechanically complex robotic arm mounted on their back. An elegant alternative is "arm-free" manipulation, where the robot balances on three legs and uses a front leg as an end-effector.
While this saves weight, it introduces a massive control challenge: the robot's "hand" is also part of its "legs," and its cameras move violently as it walks. Furthermore, existing World Action Models (WAMs)—which are powerful because they jointly predict future video frames and actions—are almost exclusively trained for stationary robot arms and are notoriously slow to run, often requiring dozens of denoising steps that make real-time quadrupedal balance impossible.
Methods
The authors propose a two-stage pipeline to turn a slow, arm-centric model into a fast, agile quadrupedal policy:
Stage 1: Cross-Morphology Adaptation and Acceleration
First, the researchers take an existing arm-centric WAM (LingBot-VA) and swap its "senses" and "muscles." They replace the arm's wrist cameras with the quadruped’s ego-centric and top-down cameras and map the 30-dimensional arm commands to a 9-dimensional command for a quadrupedal whole-body controller.
To solve the speed issue, they introduce On-Policy Self-Distillation (OPSD). A "teacher" model (the slow, high-accuracy version) supervises a "student" model (a fast version that uses only 1-2 sampling steps). Crucially, this happens "on-policy," meaning the student is trained on the exact states it encounters while actually trying to run the task, which helps it recover from its own errors.
Stage 2: Task-Specific Improvement and Merging
Imitation learning is limited by the quality of the demonstrations. For difficult tasks like opening a heavy door, the authors use Reinforcement Learning (RL) to fine-tune the teacher into "specialists." Finally, they use Multi-teacher On-Policy Distillation (M-OPD) to merge these slow specialists back into a single, fast student policy that retains the expert knowledge of all tasks.
Datasets
The authors introduce QLM-Bench (Quadrupedal Loco-Manipulation Benchmark), a MuJoCo-based environment featuring ten diverse tasks:
- Closing a fridge/oven door
- Turning a valve
- Pulling a drawer
- Opening a room door
- Pressing a button, lifting a basket, kicking a ball, and pushing/sweeping blocks.
The dataset contains 1,000 language-annotated demonstrations. Interestingly, these weren't created via tedious human teleoperation; instead, they were synthesized automatically using a large multimodal model to draft trajectories that were then refined through simulation.
Key Insights and Ideas
- Morphology is just an interface: One of the most surprising findings is that the "common sense" and world-modeling capabilities of a model trained on robot arms transfer remarkably well to a four-legged robot. By only changing the input/output layers, the model retains its ability to understand spatial relationships and language.
- On-Policy is the Secret Sauce: Distilling a diffusion-based model usually leads to a drop in success. However, by training on the student's own "streaming rollouts," the model learns to correct the small drifting errors that accumulate during fast, few-step sampling.
- Merging Specialists: Rather than training one giant model on everything at once, it is more effective to train slow experts on hard tasks and then "compress" that expertise into a fast generalist student.
Key Results
- Speed: The OPSD student achieves a 14x speedup over the original teacher, reducing inference time from 2.226s to just 0.159s per chunk.
- Accuracy: On QLM-Bench, the Stage 1 student reached a 76.7% success rate. After Stage 2 (RL and Merging), the success rate climbed to 85.0%.
- Real-World Transfer: Despite being trained entirely in simulation, the OPSD student was successfully deployed on a real Unitree Go2 Air robot, succeeding in 7 out of 30 trials across three tasks (pressing a button, kicking a ball, and pushing a box) without any real-world fine-tuning.
Limitations and Future Work
- Sim-to-Real Gap: While the zero-shot transfer worked, the success rate on the real robot (approx. 23%) is significantly lower than in simulation (85%). The authors suggest that adding real-robot data to the training loop is a necessary next step.
- Latency: Even with the 14x speedup, the full closed-loop control (including image encoding) takes 0.66s, which is still slightly slower than the 0.32s "chunk" of time the robot executes. Accelerating the visual encoder is a priority for future work.
- Task Scaling: Stage 2 currently only tunes two specific tasks with RL. Scaling this to all ten tasks (and beyond) would require more generalized reward functions.
Summary updated