Real-time Iteration Scheme for Diffusion Policy

Yufei Duan, Hang Yin, Danica Kragic

TL;DR

Real-Time Iteration for Diffusion Policy (RTI-DP) brings high-frequency control to diffusion-based robotics by "warm-starting" the denoising process. By using the previous time step's prediction as an initial guess for the current one, the authors achieve 10x to 30x speedups in inference time—reducing latency from nearly a second to just 25ms—without requiring any retraining or model distillation.

What Problem Does This Paper Address?

Diffusion models have become a gold standard for robotic manipulation because they can represent complex, multi-modal behaviors. However, they are notoriously slow. A standard Diffusion Policy (DP) starts every inference cycle from pure Gaussian noise, requiring dozens or hundreds of denoising steps to generate an action.

In robotics, this creates a "bottleneck": the robot must wait for a long computation before it can move, leading to jerky, non-smooth motions and an inability to react to high-speed changes. While existing solutions like "distillation" (compressing the model) exist, they are computationally expensive to train and often lose the diversity of behaviors that made diffusion attractive in the first place.

Methods

The authors introduce the Real-Time Iteration (RTI) scheme, a concept borrowed from optimal control, and apply it to diffusion inference.

  1. Warm-Starting (Iterative Refinement): Instead of starting from random noise at every time step, RTI-DP takes the action sequence predicted at time $t-1$, shifts it forward (removing the executed action and repeating the last one), and uses it as the starting point for time $t$.
  2. Truncated Denoising: Because the "warm-start" guess is already close to a good solution (due to the physical continuity of robot movement), the model only needs a few denoising steps (often just 1–3) to refine the action, rather than the full sequence.
  3. Handling Discrete Actions: Diffusion models struggle with abrupt changes, like a binary "grasp" command. The authors propose a scaling-based method (e.g., dividing discrete values by 10) to transform these signals into a range that the diffusion model can smoothly interpret and denoise during these truncated steps.
  4. Theoretical Guarantee: The paper provides a proof for Local Contractivity. It demonstrates that if the initial guess is sufficiently close to the truth, the errors in the denoising process will decay exponentially, ensuring the robot stays on a stable trajectory.

Datasets

The method was evaluated on three major robotic manipulation benchmarks:

  • Robomimic: Includes five distinct tasks (Can, Lift, Transport, Tool Hang, and Square) involving single and dual-robot setups and high-precision actions.
  • Push-T: A task requiring an agent to push a T-shaped block to a target using precise contact-rich interactions.
  • Multimodal Block Pushing: A benchmark testing the ability to handle multiple goals (pushing two blocks into squares in any order).

Key Insights and Ideas

  • Spatiotemporal Consistency is Key: The core "aha" moment is realizing that robots move in a continuous physical world. The best guess for what a robot should do now is almost always a slightly updated version of what it planned to do one millisecond ago.
  • Plug-and-Play Acceleration: Unlike "Consistency Models" or "Distillation," RTI-DP requires zero retraining. You can take a massive, pre-trained diffusion model and immediately run it 20x faster just by changing the inference loop.
  • Closing the Loop: Standard Diffusion Policies often run "open-loop" (predict a chunk of actions, execute them all, then predict again). RTI-DP enables a "one-action-per-prediction" strategy, allowing for continuous sensor feedback and much more responsive control.

Key Results

  • Massive Speedups: In the "Push-T" task, inference time dropped from 816ms (Standard DP) to 25ms (RTI-DP).
  • Maintained Performance: Despite the drastic reduction in computation, RTI-DP achieved success rates comparable to—and in some state-based tasks, higher than—the original full-step Diffusion Policy.
  • Superiority over Baselines: RTI-DP consistently outperformed other fast-inference methods like Streaming Diffusion Policy (SDP) and Consistency Policy (CP) in terms of the balance between success rate and speed.

Limitations and Future Work

  • Sudden Changes: The method relies on the assumption that actions change smoothly. If a task requires a sudden, unpredictable "snap" in behavior that wasn't in the previous plan, the truncated denoising might struggle to catch up in just 3 steps.
  • Parameter Tuning: While the paper provides a theoretical basis for choosing the number of denoising steps ($K'$), in practice, this value still requires some empirical tuning depending on the specific robot and task.
  • Future Directions: The authors suggest exploring how this "warm-starting" logic could be applied to other generative architectures, such as flow-matching models, to further push the boundaries of real-time robotic intelligence.

Summary updated

Abstract

Diffusion Policies have demonstrated impressive performance in robotic manipulation tasks. However, their long inference time, resulting from an extensive iterative denoising process, and the need to execute an action chunk before the next prediction to maintain consistent actions limit their applicability to latency-critical tasks or simple tasks with a short cycle time. While recent methods explored distillation or alternative policy structures to accelerate inference, these often demand additional training, which can be resource-intensive for large robotic models. In this paper, we introduce a novel approach inspired by the Real-Time Iteration (RTI) Scheme, a method from optimal control that accelerates optimization by leveraging solutions from previous time steps as initial guesses for subsequent iterations. We explore the application of this scheme in diffusion inference and propose a scaling-based method to effectively handle discrete actions, such as grasping, in robotic manipulation. The proposed scheme significantly reduces runtime computational costs without the need for distillation or policy redesign. This enables a seamless integration into many pre-trained diffusion-based models, in particular, to resource-demanding large models. We also provide theoretical conditions for the contractivity which could be useful for estimating the initial denoising step. Quantitative results from extensive simulation experiments show a substantial reduction in inference time, with comparable overall performance compared with Diffusion Policy using full-step denoising. Our project page with additional resources is available at: https://rti-dp.github.io/
BibTeX
@inproceedings{duan2025real,
  author = {Yufei Duan and Hang Yin and Danica Kragic},
  title = {{Real-time Iteration Scheme for Diffusion Policy}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2025},
  doi = {10.1109/iros60139.2025.11247391},
}

Loading related content...

Figure