In the race to make robot policies faster and more responsive, researchers have turned to complex "caching" mechanisms to skip redundant computations. But a new paper from independent researchers Abhinav Gupta and Nissan Pow asks a provocative question: What if we just did less work in the first place?
By evaluating the state-of-the-art Fast-WAM model, the authors demonstrate that simply reducing the number of denoising steps often outperforms sophisticated caching strategies in both speed and reliability.
TL;DR
The paper shows that for flow-matching robot policies like Fast-WAM, using just two denoising steps achieves the same success rate as ten steps while cutting latency by more than half. Complex cross-cycle caching (like $C^3$ache) often fails to beat this simpler baseline and proves surprisingly fragile when noise settings or sampling schedules change.
What Problem Does This Paper Address?
Modern robot policies often use diffusion or flow-matching to generate actions. These models work by starting with random noise and iteratively "denoising" it into a valid action sequence (an action chunk). The catch? Each step requires a full pass through a massive neural network. For a 6-billion-parameter model like Fast-WAM, ten steps take about 200ms—a lifetime for a robot that needs to react at 20Hz.
To solve this, researchers previously proposed caching: reusing the "residuals" (internal computations) from previous planning cycles to skip steps in the current one. While caching claims significant speedups, this paper investigates whether those gains hold up against a more optimized baseline: simply taking fewer, higher-quality denoising steps.
Methods
The authors performed a head-to-head comparison using Fast-WAM, a world action model consisting of a 5B-parameter video transformer and a 1B-parameter action transformer.
They compared two main approaches:
- Uncached Baselines: Running the model with a variable number of Euler integration steps (from 1 to 10).
- $C^3$ache Implementation: A cross-cycle cache that stores residuals from the first 7 or 8 steps of a "refresh" call and reuses them for subsequent "reuse" calls.
The study meticulously tracked both task success and inference latency on an NVIDIA RTX A6000. Crucially, they also tested how these methods reacted to changes in the time shift (a parameter that controls which parts of the noise-to-action flow the model focuses on) and noise resampling (whether the starting noise is the same or different across calls).
Datasets
The evaluation was conducted in the LIBERO simulation environment, specifically:
- LIBERO-Goal: 10 tasks involving manipulating objects to reach goal states.
- LIBERO-Long: A suite of longer-horizon tasks.
- The authors used up to 500 initial states to ensure statistical significance, employing McNemar tests and Tango scores to verify "non-inferiority."
Key Insights and Ideas
- The "Moving Baseline" Problem: Previous caching research compared cached versions of 10-step models against 10-step uncached models. However, the authors found that two steps are already sufficient to reach ~96% success. Once you adopt a 2-step baseline, the "overhead" of managing a cache (encoding images, running video experts) often makes the cache slower than just running the two steps fresh.
- The "Share of Flow" Metric: The success of a cache isn't just about how many steps you skip, but where those steps are in the trajectory. A "time shift" in the sampler can change the portion of the flow path covered by the cache. For example, 8 cached steps might cover 44% of the path in one setting (success) but 80% in another (failure).
- Noise Fragility: Caches are often "noise-locked." If the robot policy resamples noise on every call (a common practice to prevent getting stuck), the success of cached policies plummets (losing 14–40 points), whereas uncached policies remain robust.
Key Results
- Efficiency: Reducing Fast-WAM from 10 steps to 2 steps maintained a 95-96% success rate while dropping latency from 202ms to 92ms.
- Caching Overhead: A cache that keeps two fresh steps cannot be faster than two uncached steps because the fixed costs of image encoding and video processing remain identical.
- Fragility: Under Fast-WAM’s current default settings, the $C^3$ache policy’s success dropped to 63%, compared to 95% for the uncached version.
- Cost Model: The authors' cost model suggests that even an aggressive cache refreshed every four calls only saves about 11–14% of computation compared to the 2-step baseline.
Limitations and Future Work
The authors acknowledge several limitations:
- Simulation Only: All tests were performed in the LIBERO simulator; real-world hardware might introduce different latency jitters.
- Architecture Specificity: The results are specific to the Fast-WAM architecture and the LIBERO-Goal tasks.
- Fixed Delays: The study used fixed delays for asynchronous execution rather than measuring real-time concurrency.
The paper concludes with a checklist for future researchers, urging them to always compare against fewer-step baselines and to test caching strategies under the exact noise and sampler settings intended for deployment. Their code and evaluation harness are available on GitHub.
Summary updated