Robot policies that generate actions with a diffusion model (a generator that turns random noise into a sample by many small denoising steps) are now fast enough for real-time control because the many steps can be distilled into one or a few. These fast policies have no safety check, and their inputs come from cameras whose interpretation can be wrong: a vision model can misplace an obstacle or report one that is not there. This proposal builds a safety filter that fits inside the time budget of a one-step policy and that still holds when perception is wrong within a calibrated margin. The expensive non-linear arithmetic proof is done once, offline, over all obstacle placements. At run time, checking one action reduces to bounding a low-degree polynomial over a small box, which takes microseconds.
The motivation is a mismatch between three lines of work. Distillation has removed the latency barrier. One-Step Diffusion Policy raises control frequency from 1.49 Hz to 62.5 Hz, and Consistency Policy runs in about 21 ms on a laptop GPU, but both execute whatever the vision backbone implies. Formal verification of polynomial systems can prove safety, but it takes minutes. The sums-of-squares invariant method needed about 25 minutes for one 4D system at degree 8. Work on multimodal hallucination shows that the perceived geometry a verifier would check is itself unreliable. STSBench finds that a text model given true coordinates beats pixel-fed vision models (57.08% vs. 46.07%). A safety filter that trusts the perceived scene therefore certifies the wrong world.
Core Claim
A safety certificate for a fast visuomotor policy is only useful if it is robust to perception error and cheap enough to evaluate at control rate. Both requirements can be met by synthesizing, offline, a barrier certificate that is polynomial jointly in the robot state and in the obstacle parameters. A barrier certificate is a function that is non-negative exactly on safe states and cannot be driven negative in one step. With obstacle position and size treated as variables inside the proof, the online check over a whole uncertainty box of obstacle placements is a sound polynomial range bound rather than a solver call.
Concretely, let $x$ be the robot state, $u$ a proposed action, $f(x,u)$ the known one-step dynamics, $\theta$ the obstacle parameters, and $B(x,\theta)$ a polynomial with safe set $\{B \ge 0\}$. Offline we find $B$ by sums-of-squares programming so that $B(x,\theta) < 0$ whenever the robot intersects the obstacle described by $\theta$. Online, given a perception estimate turned into a box $\Theta_t$ of plausible obstacle parameters, we accept $u$ only if
$$\min_{\theta \in \Theta_t} \Big[ B(f(x,u),\theta) - (1-\gamma)\,B(x,\theta) \Big] \ge 0,$$
where $\gamma \in (0,1]$ sets how fast the system may approach the boundary. The minimum is lower-bounded soundly with Bernstein or interval bounds on the polynomial over the box. A lower bound of zero or more proves acceptance; a failed bound triggers fallback.
The guarantee is explicitly conditional. Safety holds whenever the true obstacle lies inside $\Theta_t$. We calibrate the box size on held-out frames so that it covers the true obstacle at a chosen rate. The research question is whether that conditional guarantee, at a box size that keeps the robot useful, prevents the violations that perception errors cause.
Research Program
The work has four pieces.
Offline certificate synthesis. We extend the piecewise-polynomial formulation of Adjé et al. by adding obstacle parameters as extra variables. That paper recasts non-negativity proofs as semidefinite programs and shows that raising degree tightens bounds (error bound 17.4 down to 2.02 from degree 6 to 10). It also shows that the program grows quickly with dimension. We therefore keep the certificate local, over end-effector position plus a few obstacle parameters, rather than the full arm state. We use one certificate per obstacle class and combine certificates by taking their minimum.
When the semidefinite program is too large, we refine candidate certificates instead. A cheap learner proposes candidates from simulated rollouts, and a sound checker refutes them with counterexamples. This is the learner–teacher loop of NumInv. For the checker we borrow the tangent-plane linearization of products from IC3-NRA-prove, which proves non-linear transition systems safe by repeatedly refining a linear abstraction. Neither prior tool deals with perception: they verify programs with exactly known inputs. Our contribution on this side is quantifying over the perceived quantities.
Perception uncertainty boxes. The box $\Theta_t$ comes from an external obstacle detector, with box size calibrated by split-conformal coverage on held-out labeled frames. A spurious obstacle only adds conservatism. A missed obstacle is the dangerous case, and no box covers it. We handle misses with a disagreement check. Following Adaptive Safety Probing, which found that a linear probe on OpenVLA layer 8 recovers 3D target coordinates with R² of 0.99, we train a probe that reads object geometry from the policy's own visual features. Any object the probe localizes but the detector misses is added to $\Theta_t$ with an inflated box. We will report the residual miss rate honestly as the boundary of the guarantee.
Cheap fallback. One-step generators make fallback nearly free in latency. When an action fails the check, we draw a batch of noise seeds in a single batched forward pass and take the first action that passes. Only if none passes do we fall back to the action that maximizes $B$ (a safe stop or retreat). This differs from the re-run strategy of Adaptive Safety Probing: that method re-queries the whole model sequentially, and it decides when to escalate with a statistical probe rather than a proof.
Analysis. We split every observed violation into three classes: policy stochasticity, perception error inside the box, and perception error outside the box. This tells us which component is doing the work.
Baselines
Raw policies (do-nothing reference). We run unfiltered OneDP and Consistency Policy, trained with the authors' recipes on identical demonstrations. The published reference points are OneDP-S at 84.3% average Robomimic success against a teacher's 82.9%, and a 3-step Consistency Policy at 77% on Tool Hang against 79% for the 100-step teacher, as reported in the respective papers. These set the task-success ceiling our filter must stay close to. Beating this baseline means a large reduction in safety violations at a success drop of only a few points.
Point-estimate polynomial filter. This is the same certificate with $\Theta_t$ shrunk to the detector's single estimate. It isolates the key ingredient, robustness over the uncertainty box. It runs on the same policy, detector, dynamics, and $\gamma$. Our method beats it if violations caused by perception error drop substantially while success stays comparable.
Inflated-margin linear or box filter. Each obstacle becomes an axis-aligned keep-out box enlarged by the same calibrated margin, checked with linear constraints. This is the cheapest credible competitor and tests whether non-linear arithmetic is needed at all. It receives the same calibration data and the same margin-tuning budget. Our method beats it only if, at equal violation rate, it gives higher task success, meaning it is less conservative.
Solver refinement in the style of DiffuSolve. DiffuSolve samples trajectories from a diffusion model and repairs them with a non-linear programming solver. DiffuSolve+ reaches 94.8% success on Two-Car Reach-Avoid against 64.1% for uniform initialization, as that paper reports. It is the strongest existing method that combines diffusion generation with exact constraint checking. On its own state-based benchmarks we run it with its published settings. On the vision-conditioned tasks we feed it the same detector estimates. We compare violations, success, and wall-clock latency per control step. The file has no reported per-step latency for this pipeline, so we measure it.
Probe-triggered re-run in the style of Adaptive Safety Probing. We reimplement its trigger-and-re-run loop on the same diffusion policies. It reports a 69.5% latency reduction against always-on verification and correction of 70–80% of detected failures by re-running. Beating it means fewer violations specifically on perception-caused failures at similar latency.
Why not just…?
Why not just use a standard safety filter on the detected obstacle positions? This is the most serious threat. If perception errors in these tasks are small and rare, the point-estimate filter will match ours. The point-estimate filter baseline settles the question directly. Our method is only justified if a substantial share of violations comes from in-box perception error, which the analysis split measures. STSBench's gap between coordinate-fed and pixel-fed models suggests these errors are not rare, but that is evidence from driving question answering, not manipulation. We will report the result whichever way it falls.
Why not just use linear keep-out boxes with a safety margin? Linear checks are cheaper and sound, and with generous margins they may be good enough. The case against them is conservatism. The SOS work shows coarse approximations of curved bad regions give much looser bounds, and DiffuSolve stresses that its trajectory problems are non-convex. In cluttered scenes inflated boxes can close off feasible paths. The inflated-margin baseline measures exactly this trade-off. If it matches us on the success-versus-violation curve, the non-linear certificate is unnecessary and we will say so.
Why not just re-run the policy when something looks wrong? Adaptive Safety Probing shows re-running fixes most stochastic failures. However, re-sampling with the same corrupted observation reproduces a perception-caused error. Our failure-class analysis tests whether re-running specifically fails on perception-caused violations, and on that class our robust check should win.
Why not just train the hazard away with reinforcement learning? AutoDrive-P³ cuts collisions by 40% with hierarchical reward shaping. Its authors still report occasional hallucinated objects, and it runs at 1 Hz. Reward shaping is complementary: it can make the base policy propose fewer unsafe actions, but it gives no per-action guarantee. We treat this as an orthogonal improvement rather than a competitor.
Proposed Experiments
State-based sanity check. On DiffuSolve's Two-Car Reach-Avoid and Quadrotor Navigation tasks, which have known non-linear dynamics, we first verify that the parametric certificate and online range bound are correct and fast. We measure the zero-violation rate with exact obstacle parameters and the median time to check one action.
Rendered versions of the same tasks. We render top-down images and use a learned detector, then inject controlled perception errors: position noise, scale error, spurious obstacles, and dropped obstacles. All baselines run under each error type.
Robomimic and Push-T with obstacles. We add keep-out obstacles to Robomimic Square, Tool Hang, and Push-T. The policies are OneDP and Consistency Policy conditioned on images, with both natural and injected detector errors.
Metrics. We report safety violations per episode, task success, end-to-end control frequency, fraction of steps that trigger fallback, and the three-way failure split.
Decisive checks. Four results decide the proposal. First, the filtered one-step policy must stay at or above 20 Hz, the hardware limit OneDP reports for its Franka arm. Second, it must reduce violations relative to the point-estimate filter under in-box perception error. Third, it must dominate the inflated linear filter on the success-versus-violation curve. Fourth, on perception-caused failures it must beat probe-triggered re-running. Failing the third check would mean the non-linear arithmetic is unnecessary. Failing the second would mean the hallucination robustness adds nothing. Either outcome is informative and we will report it.
References
- One-Step Diffusion Policy: Fast Visuomotor Policies via Diffusion Distillation
- Consistency Policy: Accelerated Visuomotor Policies via Consistency Distillation
- Property-based Polynomial Invariant Generation Using Sums-of-Squares Optimization
- STSBench: A Spatio-temporal Scenario Benchmark for Multi-modal Large Language Models in Autonomous Driving
- Counterexample-guided approach to finding numerical invariants
- Invariant Checking of NRA Transition Systems via Incremental Reduction to LRA with EUF
- Adaptive Safety Probing for Resource-Efficient Vision-Language-Action Models
- DiffuSolve: Diffusion-Based Solver for Non-Convex Trajectory Optimization
- $AutoDrive\text{-}P^3$: Unified Chain of Perception–Prediction–Planning Thought via Reinforcement Fine-Tuning