Continuous-Action World Models: Learning Cross-Embodiment Spatiotemporal Dynamics Priors

Liangyu Li, Qingwen LiuarXiv 2026 · arXiv

TL;DR

Continuous-Action World Models (CoAWM) represents robot actions as continuous latent functions rather than discrete command sequences. This approach allows a single model to learn from diverse robot types (cross-embodiment) and execute tasks at any control frequency, resulting in significantly smoother motion and improved energy efficiency.

What Problem Does This Paper Address?

Robot learning currently suffers from a "data fragmentation" problem. Datasets are collected using different robot bodies (embodiments), varying action interfaces (e.g., joint positions vs. end-effector poses), and inconsistent control frequencies (e.g., one dataset at 10Hz, another at 50Hz).

Standard world models treat actions as discrete "chunks" or sequences. This makes it difficult to:

  1. Transfer knowledge: A reaching motion learned on one robot doesn't easily translate to another with a different control rate.
  2. Ensure smoothness: Discrete jumps between predicted action chunks often cause "jerkiness," which is hard on robot hardware and inefficient.
  3. Adapt execution: Most models are locked into the frequency they were trained on.

CoAWM addresses these gaps by learning a shared "spatiotemporal dynamics prior"—essentially a universal library of how motions unfold over physical time, independent of the specific robot or sampling rate.

Methods

The core innovation is treating an action sequence as a continuous function $a(\tau)$ that can be queried at any time point. The authors implement this through a two-stage training process:

Stage A: Learning the Shared Prior

The authors build a "function autoencoder." * Encoder: A Mixture-of-Transformers (MoT) architecture processes timestamped trajectories. It uses specialized "experts" for position (XYZ), orientation (Q), and gripper (G) states, but allows them to attend to each other to capture coordination. * Frequency Invariance: To make the model frequency-agnostic, they use "paired rate views"—training the encoder to produce the same latent representation for the same motion sampled at two different frequencies. * Decoder: A time-queryable decoder that takes a latent vector and a specific time $t$ to output the corresponding action.

Stage B: Connecting to a World Model

The pretrained, frozen function space is then linked to a world-model backbone (based on LaWAM).

  • The model uses visual observations (head and wrist cameras), language instructions, and proprioception to predict a "latent action function" in the shared space.
  • Anchored Geometric Readout: To prevent "teleportation" jumps at the start of a new plan, the model anchors its predictions to the robot's current physical pose.
Figure 1. Overview of continuous action-function learning and deployment. Training Stage A learns shared spatiotemporal priors from typed trajectories across embodiments and temporal sampling grids. Training Stage B connects observation-conditioned latent prediction to the learned function space using latent and physical reconstruction supervision. At inference, current multimodal observations and visual features condition latent prediction; the function decoder combines the predicted latent with physical query times and observed context to produce embodiment-specific control commands.

Datasets

The paper focuses on bimanual (two-armed) manipulation.

  • Training/Evaluation: The authors utilize 16 diverse manipulation tasks (e.g., "dual bottles," "hammer," "burger assembly").
  • Validation: They used 8,192 held-out evaluation windows to test reconstruction accuracy.
  • Cross-Frequency Testing: The model was evaluated across six different execution frequencies (10, 15, 20, 30, 50, and 60 Hz) without retraining.

Key Insights and Ideas

  • Actions as Functions, Not Tokens: By representing motion as $a = f(z, t)$, the model treats "time" as a continuous input. This decouples the structure of the motion from the clock of the controller.
  • Decoupling Dynamics from Context: Stage A learns how things move (dynamics), while Stage B learns what to do based on the scene (task conditioning). This separation makes it easier to scale the model using heterogeneous data that might lack camera views or language labels.
  • The "Jerk" Reduction: Because the function space is continuous, the transitions between actions are naturally smoother. The authors found that this doesn't just look better—it actually reduces the physical work the robot has to perform.
Schematic showing how a single continuous function can be queried at different grids or retimed.

Key Results

  • High Fidelity: The function autoencoder reconstructed held-out trajectories with a position error of only 2.503 mm and a rotation error of 0.2295°.
  • Superior Smoothness: Compared to the baseline (LaWAM), CoAWM reduced translation jerk by 36.7% and rotation jerk by 54.2% while maintaining the same movement speeds.
  • Frequency Agnostic: The same policy achieved stable success rates (ranging from 73.3% to 80.7%) across all tested frequencies from 10Hz to 60Hz.
  • Energy Efficiency: The smoother motion led to a 22.6% reduction in estimated positive mechanical work across successful trials.
  • Task Success: Achieved high success rates on complex bimanual tasks, such as 94% on the "dual bottles" task.

Limitations and Future Work

  • Sustained Execution: While initial execution is much smoother, the authors noted that "jerk tails" (instability) can still emerge over longer execution windows and during frequent replanning.
  • Scale: This paper serves as a "rapid validation." The authors plan to expand the training scale significantly in future versions to include more embodiments and perform full fine-tuning.
  • Stability: Further optimization is needed to ensure stability during long-horizon tasks where the robot must continuously adjust its plan.

Summary updated

Abstract

We introduce Continuous-Action World Models (CoAWM), a framework for learning cross-embodiment spatiotemporal dynamics priors that support robot action generation across embodiments and execution frequencies. At its core is a shared latent function space that captures the spatial and temporal structure of robot actions. We learn this space through two stages. First, function pretraining brings trajectories from different embodiments and sampling frequencies into a common representation, learning shared dynamics priors through action reconstruction and consistency across temporal sampling views. By accommodating heterogeneous robot trajectories, this stage broadens the range of data sources available for pretraining and facilitates scaling to larger and more diverse datasets. Second, we connect the pretrained function space to a world-model backbone, enabling observations, language instructions, and the current robot state to condition the prediction of a latent action function. A single decoder takes any query time and returns the corresponding end-effector action. By separating shared dynamics priors from task-conditioned action selection and temporal sampling, our approach provides a unified foundation for transferring robot experience across embodiments and control frequencies.
BibTeX
@misc{li2026continuous,
  author = {Liangyu Li and Qingwen Liu},
  title = {{Continuous-Action World Models: Learning Cross-Embodiment Spatiotemporal Dynamics Priors}},
  year = {2026},
  eprint = {2610.40401},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.40401},
}
Figure