TL;DR
Continuous-Action World Models (CoAWM) represents robot actions as continuous latent functions rather than discrete command sequences. This approach allows a single model to learn from diverse robot types (cross-embodiment) and execute tasks at any control frequency, resulting in significantly smoother motion and improved energy efficiency.
What Problem Does This Paper Address?
Robot learning currently suffers from a "data fragmentation" problem. Datasets are collected using different robot bodies (embodiments), varying action interfaces (e.g., joint positions vs. end-effector poses), and inconsistent control frequencies (e.g., one dataset at 10Hz, another at 50Hz).
Standard world models treat actions as discrete "chunks" or sequences. This makes it difficult to:
- Transfer knowledge: A reaching motion learned on one robot doesn't easily translate to another with a different control rate.
- Ensure smoothness: Discrete jumps between predicted action chunks often cause "jerkiness," which is hard on robot hardware and inefficient.
- Adapt execution: Most models are locked into the frequency they were trained on.
CoAWM addresses these gaps by learning a shared "spatiotemporal dynamics prior"—essentially a universal library of how motions unfold over physical time, independent of the specific robot or sampling rate.
Methods
The core innovation is treating an action sequence as a continuous function $a(\tau)$ that can be queried at any time point. The authors implement this through a two-stage training process:
Stage A: Learning the Shared Prior
The authors build a "function autoencoder." * Encoder: A Mixture-of-Transformers (MoT) architecture processes timestamped trajectories. It uses specialized "experts" for position (XYZ), orientation (Q), and gripper (G) states, but allows them to attend to each other to capture coordination. * Frequency Invariance: To make the model frequency-agnostic, they use "paired rate views"—training the encoder to produce the same latent representation for the same motion sampled at two different frequencies. * Decoder: A time-queryable decoder that takes a latent vector and a specific time $t$ to output the corresponding action.
Stage B: Connecting to a World Model
The pretrained, frozen function space is then linked to a world-model backbone (based on LaWAM).
- The model uses visual observations (head and wrist cameras), language instructions, and proprioception to predict a "latent action function" in the shared space.
- Anchored Geometric Readout: To prevent "teleportation" jumps at the start of a new plan, the model anchors its predictions to the robot's current physical pose.
Datasets
The paper focuses on bimanual (two-armed) manipulation.
- Training/Evaluation: The authors utilize 16 diverse manipulation tasks (e.g., "dual bottles," "hammer," "burger assembly").
- Validation: They used 8,192 held-out evaluation windows to test reconstruction accuracy.
- Cross-Frequency Testing: The model was evaluated across six different execution frequencies (10, 15, 20, 30, 50, and 60 Hz) without retraining.
Key Insights and Ideas
- Actions as Functions, Not Tokens: By representing motion as $a = f(z, t)$, the model treats "time" as a continuous input. This decouples the structure of the motion from the clock of the controller.
- Decoupling Dynamics from Context: Stage A learns how things move (dynamics), while Stage B learns what to do based on the scene (task conditioning). This separation makes it easier to scale the model using heterogeneous data that might lack camera views or language labels.
- The "Jerk" Reduction: Because the function space is continuous, the transitions between actions are naturally smoother. The authors found that this doesn't just look better—it actually reduces the physical work the robot has to perform.
Key Results
- High Fidelity: The function autoencoder reconstructed held-out trajectories with a position error of only 2.503 mm and a rotation error of 0.2295°.
- Superior Smoothness: Compared to the baseline (LaWAM), CoAWM reduced translation jerk by 36.7% and rotation jerk by 54.2% while maintaining the same movement speeds.
- Frequency Agnostic: The same policy achieved stable success rates (ranging from 73.3% to 80.7%) across all tested frequencies from 10Hz to 60Hz.
- Energy Efficiency: The smoother motion led to a 22.6% reduction in estimated positive mechanical work across successful trials.
- Task Success: Achieved high success rates on complex bimanual tasks, such as 94% on the "dual bottles" task.
Limitations and Future Work
- Sustained Execution: While initial execution is much smoother, the authors noted that "jerk tails" (instability) can still emerge over longer execution windows and during frequent replanning.
- Scale: This paper serves as a "rapid validation." The authors plan to expand the training scale significantly in future versions to include more embodiments and perform full fine-tuning.
- Stability: Further optimization is needed to ensure stability during long-horizon tasks where the robot must continuously adjust its plan.
Summary updated