This summary provides a technical overview of the paper "Confidence-Modulated Safety Filtering for End-to-End Policies" by Borquez and Fisac, presented at the CoRL 2026 Science of Physical AI Safety Workshop.
TL;DR
This paper proposes a method to dynamically adjust how "cautious" a robot's safety filter is based on the real-time confidence of its underlying neural network policy. By modulating conservativeness, the system achieves high performance when the policy is confident and adds a critical safety margin when the policy—and potentially the safety filter’s own model—is operating outside its comfort zone.
What Problem Does This Paper Address?
End-to-end (E2E) neural network policies are excellent at mapping raw sensor data (like LiDAR or cameras) to actions, but they are notoriously brittle. When they encounter situations different from their training data, they can fail catastrophically without warning.
To prevent this, researchers often wrap policies in safety filters (like Control Barrier Functions). These filters monitor proposed actions and override them if they lead to a "failure set." However, safety filters have a major flaw: they rely on a model of the world (the Operational Design Domain, or ODD). If the environment changes—say, a racing track gets wet—the filter’s assumptions might break.
Current systems usually pick a fixed level of "conservativeness" for the filter. A permissive filter allows for high performance but might react too late when things go wrong. A conservative filter is safer but often "nags" the policy, overriding it even when it’s behaving correctly, which degrades performance. The authors aim to find a "best of both worlds" approach that doesn't require knowing exactly when the ODD has been breached.
Methods
The authors introduce a filter-agnostic pipeline that modulates conservativeness at runtime. The architecture consists of three main parts:
- The Task Policy ($\pi^{task}$): An E2E model (in this case, a TD3 agent) trained to perform a task.
- The Confidence Monitor ($\beta^{task}$): A separate module that scores how much the policy can be trusted. The authors use the reconstruction error of an autoencoder; if the current sensor input looks very different from the training data, the error is high, and confidence is low.
- The Graded Safety Filter: A safety filter where the "conservativeness" can be tuned by a parameter. They use a backup Control Barrier Function (CBF) where the parameter $\gamma$ (the barrier decay rate) acts as the "knob." A lower $\gamma$ makes the filter more cautious and intervention-heavy.
The core idea is a schedule that maps confidence to conservativeness: when confidence is high, the filter becomes permissive; when confidence drops, the filter automatically becomes more conservative.
Datasets
The researchers evaluated their approach using the F1TENTH autonomous racing simulator.
- Training: The policy was trained on five tracks featuring only gentle left-hand corners (max 85°).
- Evaluation: They tested on three significantly harder tracks with sharp 135° corners and right-hand turns—features the policy had never seen.
- ODD Shift: To test the filter's limits, they lowered the simulated tire friction from $\mu=1.05$ (nominal) to $\mu=0.6$ (low friction) without telling the safety filter.
Key Insights and Ideas
- Confidence as a Proxy for Model Failure: A key "aha" moment is that even if a confidence monitor can't "see" a change in physics (like friction), it often flags the geometric situations (like sharp turns) where those physical changes are most likely to cause the safety filter's model to fail.
- Performance vs. Margin: Inside the ODD, every setting is technically "safe," so modulation is purely about reclaiming performance. Beyond the ODD, where the filter's certificate is invalid, the extra margin provided by low-confidence modulation becomes a life-saver.
- No "Switch" Needed: Unlike other methods that try to detect if the robot has left its ODD to trigger a fallback, this method uses the same continuous pipeline everywhere. It doesn't need to know why it's unconfident; it just knows to be more careful.
Key Results
The modulated filter successfully combined the speed of a permissive filter with the robustness of a conservative one.
- Inside the ODD: The modulated filter matched the lap times of the most permissive fixed filter (~30.2s).
- Outside the ODD (Low Friction): The permissive filter crashed in 10 out of 60 laps. The modulated filter completed all 60 laps without a single crash, matching the safety of the conservative filter while being 0.5 seconds faster per lap.
Limitations and Future Work
- Simulation Only: These results are preliminary and based on F1TENTH simulations. Real-world dynamics and sensor noise may pose additional challenges.
- Confidence Calibration: The system's safety relies heavily on the confidence monitor. If the policy is "confident but wrong" (a common issue with neural networks), the filter might remain too permissive.
- Scaling to Foundation Models: The authors suggest this approach is particularly promising for large robot foundation models, which often have "uneven competence" across different tasks. Testing this on more complex, generalist agents is a logical next step.
Summary updated