Diffusion policies are robot controllers that produce an action sequence by starting from random noise and refining it over many steps. They handle tasks where several different actions are equally good, but they are slow. A recent fix reuses the previous plan instead of starting from noise. The controller shifts last cycle's action sequence forward by one step, adds a small amount of noise, and refines for only one to three steps. On the Push-T pushing task this cuts latency from 816 ms to 25 ms with no retraining (Real-Time Iteration Scheme for Diffusion Policy). That result, and the stability guarantee behind it, was shown only in simulation, on the same distribution the policy was trained on. On a real robot running a simulation-trained policy, each new observation can be surprising. A shallow refinement that starts from the old plan may then keep following a plan that no longer fits the world. It also commits to whichever of several possible behaviors it had chosen before.
We propose to measure this risk and to fix it cheaply. The fix: at every control step, the policy first checks how much its own denoiser wants to change the warm-started plan, and uses that as a "surprise" score. When the score is small, it keeps the cheap one-to-three-step refinement. When the score is large, it re-noises the plan more deeply, up to a full cold restart, so the sampler can leave the old plan's neighborhood. The score costs one extra network call and needs no extra model or training. The core claim is that this gate keeps most of the latency savings of temporal warm-starting and recovers the robustness of full denoising under sim-to-real shift. It should also beat the obvious fixed-schedule and reset heuristics at matched compute.
Core Claim and Why It Is Open
Two papers already warm-start diffusion planners over time, and both use a fixed noise level. RTI-DP proves that errors in the warm-started action sequence shrink exponentially into a bounded neighborhood. The proof assumes in-distribution conditioning, and every experiment is in simulation (Push-T, Robomimic, block pushing). CoDiG (Constraint-Aware Diffusion Guidance for Robotics) perturbs the previous racing plan with small noise to raise planning rate from about 0.25 Hz to 2.5 Hz. Its authors report that this warm start traps the planner in conservative local minima when obstacle geometry changes. CoDiG was trained on real demonstrations, so it says nothing about sim-to-real transfer.
That reported failure is evidence for our concern, and it also constrains the design. Any fixed small noise level will fail when the world jumps.
On the other side, NavDP (NavDP) transfers a simulation-trained diffusion navigation policy zero-shot to wheeled, legged and humanoid robots. It runs a full cold-start denoise every cycle and pays the latency. The sim-to-real benchmark of Robot Policy Evaluation for Sim-to-Real Transfer shows the size of the shift involved. Transfer typically loses 24–30% of success, and lighting or camera changes cost 30–50%.
Nobody has asked whether the warm start amplifies these drops or absorbs them. It could amplify them, because errors recycle from plan to plan. It could also absorb them, because carrying the plan forward smooths out noisy observations. Nobody has made the warm-start depth respond to the shift.
Our proposal differs from all of these in one respect. The amount of re-noising becomes a per-step decision, driven by a signal the policy already computes. RTI-DP and CoDiG fix it in advance, and NavDP has no warm start at all.
Mechanism
Let $\tilde a_t$ be the previous plan shifted by one step, with its tail repeated. Let $D_\theta(x, k)$ be the trained denoiser's estimate of the clean action sequence from input $x$ at noise level $k$. With a small probe noise level $k_0$, the surprise score is
$$ r_t = \frac{\lVert D_\theta(\tilde a_t + \sigma_{k_0}\epsilon,\ k_0;\ o_t) - \tilde a_t \rVert}{\sqrt{d}}, $$
Here $o_t$ is the current observation, $\epsilon$ is standard Gaussian noise, $\sigma_{k_0}$ is the noise scale at level $k_0$, and $d$ is the action-sequence dimension.
The intuition comes from RTI-DP's contractivity argument. When the shifted plan is near a mode of the policy's distribution given $o_t$, one denoising step barely moves it, so $r_t$ is small. When the observation has pushed the true mode away, $r_t$ is large. The executed re-noising level is then $k_t = \min(K, k_{\min} + \lceil \alpha\, r_t / \tau \rceil)$, where $K$ is the full schedule length. The parameter $\tau$ is calibrated on in-distribution simulation rollouts as a high quantile of $r_t$, and $\alpha$ sets how steeply depth grows.
A hard latency cap bounds the worst case. Above the cap, the controller executes the next action of the old plan while a deeper sample is computed. This keeps tail latency bounded, which matters on hardware.
For tasks with a discrete gripper command, we evaluate the score separately on the continuous and the scaled-discrete action channels. This tests whether surprise in the gripper channel predicts the premature-grasp failures that RTI-DP's divide-by-ten heuristic invites under real contact timing.
Research Program
The work has three parts.
The first is a diagnostic study. Policies are trained in nominal simulation and evaluated under controlled shifts. We report how success and plan-to-plan divergence change with warm-start depth. The shifts are the benchmark's V1–V5 perturbations (object placement, texture, lighting, camera pose), plus injected observation latency and sensor noise. This tells us whether fixed warm-starting is fragile at all, which the proposal needs to know before building anything.
The second part builds the gate, together with a calibration procedure for $\tau$ that uses only simulation data.
The third part transfers to hardware. A Franka Panda or UR5e runs IsaacLab-trained policies, and a physical Push-T setup runs a simulation-trained Push-T policy. We log $r_t$ alongside failure events, to check that the score rises before failures rather than after.
Baselines
The naive reference is the standard Diffusion Policy (Diffusion Policy) with full cold-start denoising at every cycle. It is the robustness ceiling, because it has no memory to mislead it, and the latency floor in the bad sense. On Push-T it reports 95% real-world success when trained on real demonstrations, and RTI-DP reports 816 ms latency for it.
The strongest existing method is RTI-DP with its published fixed one-to-three-step truncation, run from the same checkpoint. It reports 25 ms on Push-T and "comparable overall performance" on Robomimic. We found no reported result on Lacuna for RTI-DP under sim-to-real or perturbed evaluation. Its number on our shifted conditions must therefore be measured here.
All methods share one trained checkpoint per task, the same observation pipeline and the same GPU. Each method gets an equal tuning budget on unperturbed simulation only, with no tuning on shifted or real data. That last rule matters because the gate's threshold is also set only on clean simulation.
We count the gate as beating the baselines under two conditions. Under perturbation it must recover at least half of the success gap between fixed RTI-DP and cold-start Diffusion Policy. Its mean latency must stay within twice RTI-DP's.
Three ablations isolate the ingredient. The first, a fixed deeper truncation, matches the gate's mean latency with a constant noise level. It tests whether adaptivity matters or only average depth does. The second is a binary version that does a full cold restart when $r_t$ exceeds $\tau$ and a shallow refinement otherwise. The third replaces the denoiser-based score with a raw observation-change score.
Why not just…?
Why not just always cold-start, or use more fixed steps? Under hardware latency limits this is not viable. Contact tasks and moving obstacles cannot wait 816 ms. CoDiG's cold-start planner ran at about 0.25 Hz. The matched-mean-latency fixed-depth ablation is the real version of this objection. If it equals the gate, adaptivity adds nothing, and we will report that.
Why not just distill into a consistency or one-step policy? RTI-DP reports that its warm start sits on a better success-versus-speed trade-off than Consistency Policy and Streaming Diffusion Policy, with no retraining. A one-step student still has to cope with the same shift, and our gate needs no retraining. We include Consistency Policy trained on the same simulation data as a secondary baseline. Its sim-to-real behavior is unreported in the file, so it remains a possible threat.
Why not just reset whenever the camera image changes a lot? This is the most serious alternative. It is our observation-change ablation. Pixel change confuses harmless lighting shifts with task-relevant changes. Our score measures disagreement in action space, so it ignores shifts that do not change what the policy wants to do. If the observation-change version matches the denoiser score, the simpler heuristic wins and we will say so.
Why not just randomize the simulator more? Domain randomization is complementary, not a competitor. All of our policies are trained with it. Even so, the benchmark reports 30–50% drops under lighting and camera shifts, and it found ~200 photorealistic scenes beat 10,000 low-quality ones. Randomization also leaves the latency problem untouched.
Decisive Checks
The first check is the diagnostic study. If fixed RTI-DP loses no more success than cold-start Diffusion Policy under V1–V5 and injected latency, the premise fails. The finding becomes "temporal warm starts are robust to sim-to-real shift", which is still worth publishing, and the gate is unnecessary.
If fixed RTI-DP loses more, the second check is the success-versus-latency curve. The gate should dominate both fixed-depth and binary-reset baselines in perturbed simulation and on hardware.
The third check is predictive. On hardware logs, $r_t$ should rise before failures, with an area under the precision-recall curve clearly above that of the observation-change score. For gripper tasks, high gripper-channel surprise should precede premature grasps.
The metrics are task success, mean and 95th-percentile latency, and the share of steps that escalate to deep re-noising.
References
- Real-time Iteration Scheme for Diffusion Policy
- Constraint-Aware Diffusion Guidance for Robotics: Real-Time Obstacle Avoidance for Autonomous Racing
- NavDP: Learning Sim-to-Real Navigation Diffusion Policy with Privileged Information Guidance
- Robot Policy Evaluation for Sim-to-Real Transfer: A Benchmarking Perspective
- Diffusion Policy: Visuomotor Policy Learning via Action Diffusion