TL;DR
Real-Time Iteration for Diffusion Policy (RTI-DP) brings high-frequency control to diffusion-based robotics by "warm-starting" the denoising process. By using the previous time step's prediction as an initial guess for the current one, the authors achieve 10x to 30x speedups in inference time—reducing latency from nearly a second to just 25ms—without requiring any retraining or model distillation.
What Problem Does This Paper Address?
Diffusion models have become a gold standard for robotic manipulation because they can represent complex, multi-modal behaviors. However, they are notoriously slow. A standard Diffusion Policy (DP) starts every inference cycle from pure Gaussian noise, requiring dozens or hundreds of denoising steps to generate an action.
In robotics, this creates a "bottleneck": the robot must wait for a long computation before it can move, leading to jerky, non-smooth motions and an inability to react to high-speed changes. While existing solutions like "distillation" (compressing the model) exist, they are computationally expensive to train and often lose the diversity of behaviors that made diffusion attractive in the first place.
Methods
The authors introduce the Real-Time Iteration (RTI) scheme, a concept borrowed from optimal control, and apply it to diffusion inference.
- Warm-Starting (Iterative Refinement): Instead of starting from random noise at every time step, RTI-DP takes the action sequence predicted at time $t-1$, shifts it forward (removing the executed action and repeating the last one), and uses it as the starting point for time $t$.
- Truncated Denoising: Because the "warm-start" guess is already close to a good solution (due to the physical continuity of robot movement), the model only needs a few denoising steps (often just 1–3) to refine the action, rather than the full sequence.
- Handling Discrete Actions: Diffusion models struggle with abrupt changes, like a binary "grasp" command. The authors propose a scaling-based method (e.g., dividing discrete values by 10) to transform these signals into a range that the diffusion model can smoothly interpret and denoise during these truncated steps.
- Theoretical Guarantee: The paper provides a proof for Local Contractivity. It demonstrates that if the initial guess is sufficiently close to the truth, the errors in the denoising process will decay exponentially, ensuring the robot stays on a stable trajectory.
Datasets
The method was evaluated on three major robotic manipulation benchmarks:
- Robomimic: Includes five distinct tasks (Can, Lift, Transport, Tool Hang, and Square) involving single and dual-robot setups and high-precision actions.
- Push-T: A task requiring an agent to push a T-shaped block to a target using precise contact-rich interactions.
- Multimodal Block Pushing: A benchmark testing the ability to handle multiple goals (pushing two blocks into squares in any order).
Key Insights and Ideas
- Spatiotemporal Consistency is Key: The core "aha" moment is realizing that robots move in a continuous physical world. The best guess for what a robot should do now is almost always a slightly updated version of what it planned to do one millisecond ago.
- Plug-and-Play Acceleration: Unlike "Consistency Models" or "Distillation," RTI-DP requires zero retraining. You can take a massive, pre-trained diffusion model and immediately run it 20x faster just by changing the inference loop.
- Closing the Loop: Standard Diffusion Policies often run "open-loop" (predict a chunk of actions, execute them all, then predict again). RTI-DP enables a "one-action-per-prediction" strategy, allowing for continuous sensor feedback and much more responsive control.
Key Results
- Massive Speedups: In the "Push-T" task, inference time dropped from 816ms (Standard DP) to 25ms (RTI-DP).
- Maintained Performance: Despite the drastic reduction in computation, RTI-DP achieved success rates comparable to—and in some state-based tasks, higher than—the original full-step Diffusion Policy.
- Superiority over Baselines: RTI-DP consistently outperformed other fast-inference methods like Streaming Diffusion Policy (SDP) and Consistency Policy (CP) in terms of the balance between success rate and speed.
Limitations and Future Work
- Sudden Changes: The method relies on the assumption that actions change smoothly. If a task requires a sudden, unpredictable "snap" in behavior that wasn't in the previous plan, the truncated denoising might struggle to catch up in just 3 steps.
- Parameter Tuning: While the paper provides a theoretical basis for choosing the number of denoising steps ($K'$), in practice, this value still requires some empirical tuning depending on the specific robot and task.
- Future Directions: The authors suggest exploring how this "warm-starting" logic could be applied to other generative architectures, such as flow-matching models, to further push the boundaries of real-time robotic intelligence.
Summary updated