CauSyncVLA: Causal Latent World Modeling for Asynchronous Robot Control

Xu Pan, Chubin Zhang, Xingrui Yu, Runmin Lan, Zihao Wang, Wenkai Guo, Bofang Jia, Bo An, Ziwei WangarXiv 2026 · arXiv

TL;DR

CauSyncVLA introduces an asynchronous dual-system controller that allows robots to react instantly while heavy vision-language models are still "thinking." By using a Causal Latent World Model to predict current perceptual states from execution history and a "warm-starting" technique for action trajectories, it achieves higher success rates and smoother motion at over 5x the inference throughput of standard methods.

What Problem Does This Paper Address?

Modern Vision-Language-Action (VLA) models are powerful but computationally expensive. This creates a "latency-throughput" bottleneck in robotics:

  1. Synchronous Control: If the robot waits for the VLA to finish processing an image before moving, it becomes slow and stuttery.
  2. Naive Asynchronous Control: If the robot keeps moving while the VLA is thinking, it ends up acting on "stale" information—the image the model is processing no longer matches the robot's current physical state.

This misalignment leads to "boundary jitter" (where new plans don't match old ones) and failures in contact-rich tasks like grasping, where millimeters and milliseconds matter.

Figure 1. Coupled prediction and correction in asynchronous control. Left: Synchronous control stalls for vision, whereas naive asynchronous control acts on stale views with boundary jitter. Center: System 2 publishes KV anchors while System 1 propagates KV memory from executed history and refines plans in one frozen-expert PFWS step. Right: Control trade-offs and LIBERO-10 gains.

Methods

The authors propose CauSyncVLA, which splits the workload into two systems:

  • System 2 (Slow): A large Vision-Language Model (VLM) that periodically processes images to create "KV anchors"—compressed snapshots of perceptual memory.
  • System 1 (Fast): A lightweight Causal Latent World Model that runs at high frequency. It takes the most recent (but slightly old) KV anchor and the robot's actual movement history (states and actions) to predict what the perceptual memory should look like right now.

To ensure these predictions are useful for control, they use Action-Sensitive Distillation. Instead of just trying to reconstruct the latent features perfectly, the model is trained to minimize the difference in the actions the robot would take. This focuses the model's "attention" on the most control-relevant parts of the scene.

Finally, they use Predictive Flow Warm-Starting (PFWS). Instead of generating a new action plan from scratch (random noise), they take the unexecuted tail of the previous plan, shift it forward, and refine it in a single step. This ensures motion continuity and prevents the robot from "jumping" between different movement strategies.

Figure 2. Architecture and training of CauSyncVLA. (a) Executed history H𝑎:𝑡propagates delayed KV anchor 𝑧𝑎into predicted memory ˆ𝑧𝑡via 𝑊𝜃, while prior plan 𝐴𝑡−𝐶initializes one-step frozen-expert refinement R𝜓. (b) History H𝑎:𝑡modulates KV tokens through AdaLN blocks, and adding residual 𝛿𝑧𝑡to 𝑧𝑎gives ˆ𝑧𝑡. (c) Shifted prior Q𝑡interpolates with noise 𝜖into 𝑥𝜌for one Euler step under ˆ𝑧𝑡. (d) Fresh 𝑧𝑡and predicted ˆ𝑧𝑡query frozen endpoint map 𝐷𝜓under shared noise, flow time, and state to train 𝑊𝜃via LKV and Lact.

Datasets

The model was evaluated on:

  • LIBERO-10: A standard benchmark for 10 different manipulation tasks (e.g., "stack the green cup on the brown cup").
  • LIBERO-Plus: A more challenging version with 2,519 variants involving perturbations in lighting, object layout, and camera viewpoints.
  • Real-Robot Trials: Conducted on a 7-DoF Franka FR3 arm for tasks like cup stacking, fruit placement, and vegetable sorting.

Key Insights and Ideas

  • Execution as a Reference: The robot's own movement history is a high-fidelity "clock" that can be used to advance perceptual memory even when the cameras are lagging.
  • Latent over Pixel Prediction: Predicting the internal "KV" states of a VLM is much more efficient than trying to predict future video pixels, which often contains irrelevant background noise.
  • Control Sensitivity Matters: Not all errors in perceptual memory are equal. An error in the position of a distant object might not matter, but a tiny error in the gripper's position relative to a cup is catastrophic. Action-sensitive distillation captures this geometric sensitivity.
  • Warm-Starting for Smoothness: Reusing the previous plan as a "prior" acts as a natural regularizer, suppressing the jitter common in diffusion-based or flow-based policies.

Key Results

  • Performance: On LIBERO-10, CauSyncVLA reached a 93.8% success rate, outperforming the 10-step synchronous baseline (91.2%).
  • Speed: It delivered 5.04× higher inference throughput on a shared GPU, allowing for much faster closed-loop reactions.
  • Accuracy: The Causal Latent World Model reduced downstream action error by 10.5× compared to using stale anchors.
  • Robustness: In real-world trials, the system halved response latency (from ~72ms to ~13ms) and significantly improved motion smoothness (measured by acceleration RMS).
Downstream action error across anchor staleness. CauSyncVLA maintains much lower error as the visual anchor ages compared to using a stale anchor.

Limitations and Future Work

  • Architecture Specificity: The current implementation requires a VLA with a compatible "prefix conditioning" interface (like those based on the OpenVLA or Pi-0 architectures).
  • Exogenous Motion: While the model handles robot-induced changes well, it cannot "see" sudden changes in the environment that the robot didn't cause (e.g., a human moving an object) until the next slow visual refresh.
  • Future Directions: The authors suggest exploring "entropy-triggered" visual refreshes (asking for a new image only when the world model is uncertain) and integrating high-frequency visuo-tactile sensing for even better contact handling.

Summary updated

Abstract

Vision-Language-Action (VLA) policies must react while visual inference is still running. Overlapping computation and execution reduces waiting, yet leaves perceptual memory stale and successive action chunks disconnected. We present CauSyncVLA, an asynchronous dual-system controller that aligns perceptual-memory prediction and trajectory correction with actual robot execution. Rather than forecasting pixels or physical states, a Causal Latent World Model predicts current perceptual memory from delayed visual anchors and executed state-action history, supervised through action-sensitive distillation to retain the frozen expert's control response. Conditioned on this updated memory, Predictive Flow Warm-Starting (PFWS) aligns the unexecuted action horizon for single-step flow correction without extra training. On LIBERO-10, CauSyncVLA reaches 93.8% success at 5.04$\times$ inference throughput, improving closed-loop robustness and robot motion smoothness.
BibTeX
@misc{pan2026causyncvla,
  author = {Xu Pan and Chubin Zhang and Xingrui Yu and Runmin Lan and Zihao Wang and Wenkai Guo and Bofang Jia and Bo An and Ziwei Wang},
  title = {{CauSyncVLA: Causal Latent World Modeling for Asynchronous Robot Control}},
  year = {2026},
  eprint = {2611.25667},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.25667},
}
Figure