This paper presents CAST (Certified Action Selection at Test time), a framework for steering "frozen" robot diffusion policies to satisfy new, real-world constraints without the need for expensive retraining. By shifting from "editing" actions to "selecting" from a menu of safe alternatives, the authors demonstrate significant improvements in task success across manipulation and navigation tasks.
TL;DR
Instead of using a safety filter to "fix" a single robot action—which often breaks the task—CAST samples multiple potential actions from a generative diffusion policy, certifies them for safety, and then selects the best one using a critic. This approach maintains the policy's "viability," ensuring the robot doesn't just stay safe for one step, but avoids future dead ends where no safe actions remain.
What Problem Does This Paper Address?
Diffusion policies have become a gold standard for learning complex robot skills from human demonstrations. However, these policies are typically "frozen" after training. When deployed, they often encounter constraints never seen in the training data: a new obstacle on the table, a person walking by, or a strict energy budget.
The traditional solution is a safety filter, which takes the policy's proposed action and "edits" it (e.g., via a projection) to satisfy a constraint. The problem? If the edit is large, it can push the robot into a state where the policy has no data. The robot then "stalls" or fails the task because it doesn't know how to continue from this new, edited state. The authors argue that for generative models like diffusion, we shouldn't be editing actions; we should be leveraging the model's ability to propose many diverse, valid ways to complete a task.
Methods
The core algorithm, CAST, operates in three steps at every decision point:
- Sample: Draw $N$ (typically 8) "action chunks" from the frozen diffusion policy.
- Certify: Check each chunk against a safety certificate (like a Control Barrier Function). This ensures that even if the robot follows the chunk and then has to emergency brake, it remains safe.
- Select: From the subset of certified safe actions, pick the one that a learned value ensemble (critic) ranks highest for task success.
The authors extend this to CAST-V (Viability). They recognize that an action might be safe now but lead to a "dead end" where the policy will have zero safe options in the next few seconds. CAST-V uses short internal simulations (or a learned model, CAST-VL) to estimate the "viability" of an action—the probability that the policy will continue to find safe samples in the future.
The paper also provides a theoretical backbone, using Conformal Prediction to calibrate safety margins and proving that "selecting" is mathematically more "faithful" to the original policy than "filtering."
Datasets
The authors evaluated CAST across several challenging benchmarks:
- Robomimic (Can & Square): Standard robot manipulation tasks using human demonstrations. They introduced new runtime constraints like moving obstacles, "no-fly" zones, and effort budgets.
- Planar Quadrotor: A simulated drone that must navigate around obstacles while staying within an energy budget.
- MuJoCo Humanoid: A bipedal walker tasked with moving forward under strict torque limits.
- Unicycle: A simple navigation agent used to test complex "blocked route" scenarios.
Key Insights and Ideas
- The "Menu" Advantage: Diffusion policies are inherently stochastic. Instead of treating this as noise to be suppressed, CAST treats it as a "menu" of diverse strategies. If the first choice is blocked by an obstacle, the fifth choice might go around it naturally.
- Certify, Then Select: Most prior work tries to rank actions and then filter the winner. The authors show that certifying first is crucial. It prevents the critic from "cheating" by picking high-reward but unsafe actions that the filter then ruins.
- Viability is the Real Bottleneck: One-step safety is easy; long-term viability is hard. A robot might enter a narrow corridor that is safe for now but offers no way out if a pedestrian enters from the other side. CAST-V explicitly looks for these "safe but doomed" states.
- Faithfulness Matters: Mathematically, a safety filter projection can move an action completely outside the "support" of the policy (into territory the robot doesn't understand). Selection, by definition, stays within what the policy knows how to do.
Key Results
- Manipulation: On the Robomimic "Can" task with a moving obstacle, CAST achieved a 90% joint success rate, compared to just 61% for a standard safety filter.
- Navigation: In "dead-end" scenarios (where one path is blocked), CAST-V reached 99% success, whereas standard CAST (without viability) dropped to 81% because it frequently blundered into the blocked route.
- Efficiency: Despite sampling multiple actions, CAST is highly efficient, taking only about 15ms per decision on standard hardware, making it viable for real-time robotics.
- Zero-Shot Adaptation: CAST-VL (the learned version) was able to adapt to 500 unseen obstacle configurations per policy with high success, without any retraining of the base diffusion model.
Limitations and Future Work
- Policy Support: CAST cannot "invent" new behaviors. If the frozen policy never learned to go left, and the right path is blocked, CAST will fail. It is limited by the diversity of the original training data.
- Critic Dependency: The system relies on a good task critic. If the critic is poorly trained, the robot might pick safe but useless actions.
- Passive Safety: The current safety guarantees assume the "hazards" (like pedestrians) ignore the robot. Future work could involve more interactive safety certificates for multi-agent environments.
- Computational Cost of Viability: While CAST is fast, the simulation-based CAST-V is significantly more expensive. The learned CAST-VL mitigates this but requires a dataset of previous logs.
Summary updated