Abstract
Diffusion Policies have demonstrated impressive performance in robotic manipulation tasks. However, their long inference time, resulting from an extensive iterative denoising process, and the need to execute an action chunk before the next prediction to maintain consistent actions limit their applicability to latency-critical tasks or simple tasks with a short cycle time. While recent methods explored distillation or alternative policy structures to accelerate inference, these often demand additional training, which can be resource-intensive for large robotic models. In this paper, we introduce a novel approach inspired by the Real-Time Iteration (RTI) Scheme, a method from optimal control that accelerates optimization by leveraging solutions from previous time steps as initial guesses for subsequent iterations. We explore the application of this scheme in diffusion inference and propose a scaling-based method to effectively handle discrete actions, such as grasping, in robotic manipulation. The proposed scheme significantly reduces runtime computational costs without the need for distillation or policy redesign. This enables a seamless integration into many pre-trained diffusion-based models, in particular, to resource-demanding large models. We also provide theoretical conditions for the contractivity which could be useful for estimating the initial denoising step. Quantitative results from extensive simulation experiments show a substantial reduction in inference time, with comparable overall performance compared with Diffusion Policy using full-step denoising.
Introduction
Diffusion models have become a gold standard for robotic manipulation because they can represent complex, multi-modal behaviors. However, they are notoriously slow. A standard Diffusion Policy (DP) starts every inference cycle from pure Gaussian noise, requiring dozens or hundreds of denoising steps to generate an action. In robotics, this creates a "bottleneck": the robot must wait for a long computation before it can move, leading to jerky, non-smooth motions and an inability to react to high-speed changes. While existing solutions like "distillation" (compressing the model) exist, they are computationally expensive to train and often lose the diversity of behaviors that made diffusion attractive in the first place.
Real-Time Iteration for Diffusion Policy (RTI-DP) brings high-frequency control to diffusion-based robotics by "warm-starting" the denoising process. By using the previous time step's prediction as an initial guess for the current one, we achieve 10x to 30x speedups in inference time—reducing latency from nearly a second to just 25ms—without requiring any retraining or model distillation.
Methods
The RTI-DP approach is inspired by the Real-Time Iteration (RTI) scheme from optimal control. It leverages the spatiotemporal consistency of robotic movement to accelerate inference.
- Warm-Starting (Iterative Refinement): Instead of starting from random noise at every time step, RTI-DP takes the action sequence predicted at time $t-1$, shifts it forward (removing the executed action and repeating the last one), and uses it as the starting point for time $t$.
- Truncated Denoising: Because the "warm-start" guess is already close to a good solution, the model only needs a few denoising steps (often just 1–3) to refine the action, rather than the full sequence.
- Handling Discrete Actions: Diffusion models struggle with abrupt changes, like a binary "grasp" command. We propose a scaling-based method (e.g., dividing discrete values by 10) to transform these signals into a range that the diffusion model can smoothly interpret and denoise during these truncated steps.
- Theoretical Guarantee: We provide a proof for Local Contractivity, demonstrating that if the initial guess is sufficiently close to the truth, the errors in the denoising process will decay exponentially, ensuring stable trajectories.
Experiments and Datasets
The method was evaluated on three major robotic manipulation benchmarks:
- Robomimic: Includes five distinct tasks (Can, Lift, Transport, Tool Hang, and Square) involving single and dual-robot setups.
- Push-T: A task requiring an agent to push a T-shaped block to a target using precise contact-rich interactions.
- Multimodal Block Pushing: A benchmark testing the ability to handle multiple goals.
The core insight is that the best guess for what a robot should do now is almost always a slightly updated version of what it planned to do one millisecond ago. Unlike "Consistency Models" or "Distillation," RTI-DP requires zero retraining, allowing for plug-and-play acceleration of massive, pre-trained models.
Key Results
In the "Push-T" task, inference time dropped from 816ms (Standard DP) to 25ms (RTI-DP). Despite this drastic reduction, RTI-DP achieved success rates comparable to—and in some state-based tasks, higher than—the original full-step Diffusion Policy. RTI-DP consistently outperformed other fast-inference methods like Streaming Diffusion Policy (SDP) and Consistency Policy (CP) in the balance between success rate and speed.
Conclusion
RTI-DP enables a "one-action-per-prediction" strategy, allowing for continuous sensor feedback and responsive control. While it relies on the assumption of smooth action changes and may require empirical tuning of the denoising steps ($K'$), it offers a powerful way to deploy large diffusion models in real-time. Future work will explore applying this warm-starting logic to other generative architectures like flow-matching models.
Summary updated