CoRe-WAM: Correspondence-Aligned Temporal Residuals for World Action Models

Bin Zhou, Jialong Liu, Jianan Wang, Changhao Chen, Kani ChenarXiv 2026 · arXiv

TL;DR

CoRe-WAM introduces a parameter-efficient temporal interface called TraceDelta that allows robotic world-action models to accurately track scene changes over time. By using dense correspondence to align historical visual features with the current frame before computing temporal differences, it ensures the robot tracks actual object evolution rather than pixel-grid shifts. This simple twist yields significant performance gains in both simulation benchmarks and real-world robotic manipulation tasks.

What Problem Does This Paper Address?

For a robot to successfully manipulate objects, it needs to understand how a scene changes as a result of its actions. While comparing current and past video frames seems like an obvious way to capture this evolution, doing so on a fixed pixel grid fails when the camera or objects move.

As shown in Figure 1, a specific coordinate $(x, y)$ might contain a robot gripper at time $t-1$ but an empty table at time $t$. A naive mathematical difference between these two frames mixes up the evolution of an object with a change in spatial occupancy. This "mismatched content" problem introduces noise, making it difficult for Vision-Language-Action (VLA) models to extract meaningful temporal cues.

Why align history before differencing? Naive grid differences compare mismatched content when objects or cameras move.

Methods

The authors propose CoRe-WAM (Correspondence-Aligned Temporal Residuals for World Action Models), featuring a novel TraceDelta module. Instead of forcing the policy network to figure out pixel movement, TraceDelta establishes correspondence before computing visual changes through three distinct steps:

  1. Align History (Feature Transport): It uses a frozen tracking model (Trace Anything) to map a point $p$ in the current frame back to its historical counterpart $q$. It then uses bilinear sampling to transport the historical visual features to the current token locations.
  2. Compare Features: It computes a signed difference between the current features and the newly aligned historical features within a shared, pretrained Vision-Language Model (VLM) feature space.
  3. Inject Change: A lightweight adapter maps these differences into the policy interface. Crucially, a binary validity mask gates this residual—ensuring that invalid tracking coordinates contribute nothing—before adding it directly to the head-camera tokens.
CoRe-WAM architecture and the three-step TraceDelta module.

CoRe-WAM is highly parameter-efficient. Built on top of the Motus world-action model, it keeps the massive pretrained backbone weights frozen, optimizing only 1.59 million parameters (about 0.02% of the total policy) via action-side LoRA and the temporal adapter.

Datasets

The model was trained and evaluated on:

  • RoboTwin 2.0: A simulation benchmark consisting of 50 diverse bimanual robotic manipulation tasks (e.g., hammering, object handover, opening a laptop, and stacking). The training set included 2,386 clean episodes (19,088 samples).
  • Real-World Deployment: Evaluated on an Astribot S1 physical dual-arm robot for both in-distribution and out-of-distribution (OOD) tasks.
Examples of task-relevant scene evolution across four RoboTwin tasks.

Key Insights and Ideas

  • History can hurt without alignment: One of the paper's most striking findings is that "Raw Delta" (naive grid-based differencing) actually degrades performance on many tasks compared to a model that only looks at the current frame. Historical data is only useful if it is semantically aligned.
  • Tracking as an operator, not a token: Instead of feeding raw trajectory coordinates or motion vectors directly into the transformer policy as extra tokens, tracking is used purely as a transport operator to route features. This keeps the policy's token sequence length compact and focused.
  • Cross-architecture transferability: The TraceDelta design isn't locked into a single architecture. When integrated into a completely different baseline (StarVLA), it delivered a massive performance boost, proving that the principle of alignment-before-differencing is universally beneficial for robot learning.

Key Results

  • RoboTwin 2.0 Simulation: CoRe-WAM achieved a 92.22% clean success rate across all 50 tasks, outperforming the Motus baseline by 3.56 percentage points (pp) and Raw Delta by 2.06 pp. Under randomized conditions, it maintained an 89.60% success rate.
  • StarVLA Integration: Adding TraceDelta to StarVLA boosted its clean success rate from 58.10% to 67.62% (+9.52 pp).
  • Physical Deployment: When deployed on the Astribot S1 robot, CoRe-WAM achieved a 93.3% success rate (28/30) on an in-distribution plush-toy placement task and an 83.3% success rate (25/30) on an OOD multi-fruit sorting task.
Physical setup and deployment of CoRe-WAM on the Astribot S1 robot.

Limitations and Future Work

The primary bottleneck of CoRe-WAM is its computational overhead. Running online correspondence estimation and temporal feature processing at inference time introduces a latency of approximately 510 ms per inference step (excluding network communication). This lag can limit the robot's responsiveness in highly dynamic environments.

The authors note that future work will focus on developing more lightweight correspondence estimators and implementing feature-reuse strategies to minimize latency and improve closed-loop control.

Summary updated

Abstract

Comparing current and past observations helps robots understand scene changes and select subsequent actions during manipulation. However, comparing visual features at the same image location can mix different scene content when objects or the camera move. We introduce CoRe-WAM, a world-action model that incorporates correspondence-aligned visual changes through a parameter-efficient temporal interface. Its TraceDelta module uses correspondences from a frozen tracking model to transport historical visual features to current locations before computing signed differences in a shared pretrained feature space. Correspondence thus determines which historical content is compared with the present, rather than entering the policy as a separate trajectory representation. A lightweight adapter converts these differences into validity-gated residuals that supplement current visual conditioning, allowing the policy to use recent changes alongside current-scene information. Built on Motus, CoRe-WAM keeps the pretrained backbone weights frozen and optimizes 1.59 million parameters. With a 5,000-update adaptation budget, CoRe-WAM achieves 92.22% clean success across 50 RoboTwin 2.0 tasks, 3.56 percentage points above Motus; on randomized evaluation, it achieves 89.60% success, a 2.58-point gain. Integrating TraceDelta into a StarVLA-based policy improves clean success from 58.10% to 67.62%, supporting transfer of the temporal interface beyond Motus.
BibTeX
@misc{zhou2026core,
  author = {Bin Zhou and Jialong Liu and Jianan Wang and Changhao Chen and Kani Chen},
  title = {{CoRe-WAM: Correspondence-Aligned Temporal Residuals for World Action Models}},
  year = {2026},
  eprint = {2609.27314},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2609.27314},
}
Figure