TL;DR
CoRe-WAM introduces a parameter-efficient temporal interface called TraceDelta that allows robotic world-action models to accurately track scene changes over time. By using dense correspondence to align historical visual features with the current frame before computing temporal differences, it ensures the robot tracks actual object evolution rather than pixel-grid shifts. This simple twist yields significant performance gains in both simulation benchmarks and real-world robotic manipulation tasks.
What Problem Does This Paper Address?
For a robot to successfully manipulate objects, it needs to understand how a scene changes as a result of its actions. While comparing current and past video frames seems like an obvious way to capture this evolution, doing so on a fixed pixel grid fails when the camera or objects move.
As shown in Figure 1, a specific coordinate $(x, y)$ might contain a robot gripper at time $t-1$ but an empty table at time $t$. A naive mathematical difference between these two frames mixes up the evolution of an object with a change in spatial occupancy. This "mismatched content" problem introduces noise, making it difficult for Vision-Language-Action (VLA) models to extract meaningful temporal cues.
Methods
The authors propose CoRe-WAM (Correspondence-Aligned Temporal Residuals for World Action Models), featuring a novel TraceDelta module. Instead of forcing the policy network to figure out pixel movement, TraceDelta establishes correspondence before computing visual changes through three distinct steps:
- Align History (Feature Transport): It uses a frozen tracking model (Trace Anything) to map a point $p$ in the current frame back to its historical counterpart $q$. It then uses bilinear sampling to transport the historical visual features to the current token locations.
- Compare Features: It computes a signed difference between the current features and the newly aligned historical features within a shared, pretrained Vision-Language Model (VLM) feature space.
- Inject Change: A lightweight adapter maps these differences into the policy interface. Crucially, a binary validity mask gates this residual—ensuring that invalid tracking coordinates contribute nothing—before adding it directly to the head-camera tokens.
CoRe-WAM is highly parameter-efficient. Built on top of the Motus world-action model, it keeps the massive pretrained backbone weights frozen, optimizing only 1.59 million parameters (about 0.02% of the total policy) via action-side LoRA and the temporal adapter.
Datasets
The model was trained and evaluated on:
- RoboTwin 2.0: A simulation benchmark consisting of 50 diverse bimanual robotic manipulation tasks (e.g., hammering, object handover, opening a laptop, and stacking). The training set included 2,386 clean episodes (19,088 samples).
- Real-World Deployment: Evaluated on an Astribot S1 physical dual-arm robot for both in-distribution and out-of-distribution (OOD) tasks.
Key Insights and Ideas
- History can hurt without alignment: One of the paper's most striking findings is that "Raw Delta" (naive grid-based differencing) actually degrades performance on many tasks compared to a model that only looks at the current frame. Historical data is only useful if it is semantically aligned.
- Tracking as an operator, not a token: Instead of feeding raw trajectory coordinates or motion vectors directly into the transformer policy as extra tokens, tracking is used purely as a transport operator to route features. This keeps the policy's token sequence length compact and focused.
- Cross-architecture transferability: The TraceDelta design isn't locked into a single architecture. When integrated into a completely different baseline (StarVLA), it delivered a massive performance boost, proving that the principle of alignment-before-differencing is universally beneficial for robot learning.
Key Results
- RoboTwin 2.0 Simulation: CoRe-WAM achieved a 92.22% clean success rate across all 50 tasks, outperforming the Motus baseline by 3.56 percentage points (pp) and Raw Delta by 2.06 pp. Under randomized conditions, it maintained an 89.60% success rate.
- StarVLA Integration: Adding TraceDelta to StarVLA boosted its clean success rate from 58.10% to 67.62% (+9.52 pp).
- Physical Deployment: When deployed on the Astribot S1 robot, CoRe-WAM achieved a 93.3% success rate (28/30) on an in-distribution plush-toy placement task and an 83.3% success rate (25/30) on an OOD multi-fruit sorting task.
Limitations and Future Work
The primary bottleneck of CoRe-WAM is its computational overhead. Running online correspondence estimation and temporal feature processing at inference time introduces a latency of approximately 510 ms per inference step (excluding network communication). This lag can limit the robot's responsiveness in highly dynamic environments.
The authors note that future work will focus on developing more lightweight correspondence estimators and implementing feature-reuse strategies to minimize latency and improve closed-loop control.
Summary updated