Robots that plan by looking ahead through a search tree can enforce safety in a simple way. Before searching, they remove any first move (a "root action") that a learned predictor judges too likely to end in a crash, and then spend the whole compute budget on the moves that remain. In simulation this works well. When a planner is moved to a real robot, two things break at once. Real sensors are noisier than simulated ones, which inflates the planner's uncertainty about where it is, so the filter either deletes every action and the robot freezes, or it trusts a predictor that is now wrong. The usual repair, adjusting the risk threshold online by watching whether failures actually happen, needs real crashes to learn from. That is exactly what a physical deployment cannot afford.
We propose to calibrate the root-action filter on a signal the robot observes at every step without crashing: its realized safety margin, such as measured clearance to the nearest obstacle minus a buffer. A predictor trained in simulation forecasts a lower bound on the minimum margin each root action will produce over the next few steps. After execution the robot measures the margin it actually got and nudges the bound's confidence level up or down, with no real-world failures and no real-world retraining. Because safety now lives in this filter, the task reward can shrink to plain goal progress, which removes the hand-tuned safety penalty weights that sim-to-real practitioners spend so much effort on.
Core Claim
A filter calibrated online on dense margin feedback transfers zero-shot from simulation to a noisy real robot. It keeps the failure rate near a user-chosen budget and avoids the freezing that support-based pruning suffers under real sensor noise, and it does so with only a goal-progress reward. The mechanism that matters is the calibration signal: margins instead of failures. The rest of the planner reuses existing machinery.
The closest work is ConstrainedZero. It trains a three-headed network (policy, value, failure probability), filters root actions whose predicted failure probability exceeds a budget Δ, and adapts Δ online with adaptive conformal inference, meaning a running correction that raises or lowers the threshold according to recent prediction misses. That paper provides our planner. It also supplies the evidence that calibration is essential: without online adaptation, LightDark failure rates jumped from 0.01 to 0.66. The limitation we target is the feedback source. ConstrainedZero adapts on observed failures, in simulation only, and with idealized Gaussian observation noise. At Δ = 0.01, a correction step needs roughly one failure per hundred episodes, which is not a workable schedule on hardware.
The second anchor is Safe POMDP Online Planning among Dynamic Agents via Adaptive Conformal Prediction. It prunes search branches that leave conformal "winning regions" around predicted pedestrians and checks every state in the belief support, the full set of states the robot might occupy. That paper motivates the problem: support-based checks over-prune once observation noise widens the support. It also constrains our design, because we replace support checks with particle-weighted quantiles.
Mechanism
Let $b_t$ be the robot's belief at time $t$, represented as weighted particles. Let $m_{t+k}$ be the signed margin $k$ steps later, where a negative value means a constraint violation. A network head trained in simulation outputs $\hat q_\alpha(b_t,a)$, a predicted $\alpha$-quantile of the minimum margin $\min_{k\le H} m_{t+k}$ after taking root action $a$. The planner keeps only actions with $\hat q_{\alpha_t}(b_t,a) \ge 0$. After $H$ steps it observes the realized minimum margin and records a miss $e_t = 1$ if that margin fell below the prediction, $0$ otherwise. It then updates the working quantile level:
$$\alpha_{t+1} = \alpha_t + \gamma\,(\alpha^\star - e_{t-H}),$$
where $\alpha^\star$ is the user's target miss rate and $\gamma$ is a small step size. This is the same running correction that ConstrainedZero applies to Δ, with delayed margin misses in place of failures.
Two consequences follow. First, the feedback is dense: a margin miss is defined at every executed step, whether or not anything bad happened. Second, the correction is two-sided. If real sensor noise has made the simulation-trained predictor too pessimistic, misses become rare, $\alpha_t$ rises, the bound loosens, and the filter stops over-pruning. This is our answer to planner paralysis. If the predictor is too optimistic, the bound tightens.
Any actual failure implies a margin miss for the executed action, so the long-run failure rate is at most the long-run miss rate. Two caveats temper this. The guarantee is marginal, holding on average over time, which is the same limitation noted for Conformal Policy Control. And misses are observed only for the action actually selected, so we measure this selection effect empirically rather than assume it away.
When every root action is filtered, the planner does not deadlock. It executes the action with the largest predicted margin. We report how often this fallback fires as a direct measure of freezing.
The predictor is trained in simulation under randomized, structured sensor artifacts, including dropout, biased drift, latency jitter, and LiDAR-style edge returns, following the artifact catalogue in Pokle et al.. The task reward is goal progress only.
Research Program
We first build margin-calibrated Δ-MCTS on top of the ConstrainedZero planner, keeping its CC-PUCT selection rule (its upper-confidence rule for choosing which branch to explore under the chance constraint). The only change is that the failure head is replaced by a margin-quantile head. We then run three studies.
The first is a controlled noise-shift study. The planner is trained under one noise model and deployed under a harder, non-Gaussian one, in extended LightDark (localization under position-dependent observation noise) and in the ETH and Grand Central pedestrian gridworlds with corrupted pedestrian observations.
The second is a reward-stripping study. We ask whether a goal-progress-only reward combined with the filter matches a carefully hand-tuned multi-term reward combined with soft penalties. These are the multi-term balancing difficulties documented in Kolter et al. and Tang et al..
The third is zero-shot hardware transfer on a 1/10-scale LiDAR race car with a discretized steering and throttle set, the platform SPiDR used. The discretization respects ConstrainedZero's restriction to discrete actions.
Baselines
The naive reference is standard domain randomization over dynamics and sensor noise, with a model-free policy trained on a hand-tuned soft safety penalty. On hardware, SPiDR reports that this approach produced 6.2 times more safety violations than SPiDR on the RaceCar and 26.1 times more on the Unitree Go1.
The strongest existing zero-shot method on our hardware platform is SPiDR itself. It adds an ensemble-disagreement penalty to the training cost and reports "100% constraint satisfaction in the real world." We run it with the same simulator, noise randomization, and environment steps, and tune its penalty weight λ with the same grid budget we give our step size γ. Matching its constraint satisfaction while achieving higher task progress, measured as lap time or goal completion, and fewer freezing events counts as beating it. That paper itself notes that large λ causes freezing.
The strongest planning baseline is ConstrainedZero with its original failure-based adaptation. It has the same network size and the same search budget, trained on the same noisy simulation. In simulation its reported LightDark failure rate is 0.01 ± 0.01 against a target of 0.01. Beating it means reaching comparable failure rates under shifted noise with fewer adaptation episodes, and with less over-pruning where failures are too rare to drive its threshold.
The ACP-POMCP support-based filter is the reference on pedestrian data. It reports a 97.4% safety rate on ETH against 89.3% unshielded, 95.1% on Grand Central, and about 0.07 seconds of overhead per step. We re-run it on both clean and corrupted observations.
The ablation that isolates our key ingredient keeps our margin head and changes only the calibration: static threshold, failure-event feedback, or margin feedback. Everything else is held fixed.
Why not just…?
Why not just use ConstrainedZero as is, trained with sensor-noise randomization? This is the real threat. If the randomized noise covers reality, a static or failure-adapted threshold may be enough. Our noise-shift study deliberately deploys outside the training noise distribution and counts how many failures each variant needs before its failure rate settles. If failure-based adaptation matches ours within the same number of episodes, the margin idea adds nothing, and we will report that.
Why not just put the uncertainty penalty into training, as SPiDR does? SPiDR is simple and works on hardware. However, its safety is fixed at training time, its λ is a manual knob, and it cannot react to deployment-time noise it never saw. The hardware comparison under an unmodeled sensor artifact, such as added LiDAR occlusion or latency, tests whether online calibration matters in practice.
Why not just use a reactive reachability shield, as in Sim-to-Lab-to-Real? That approach reached a 76.7% success rate and an 86.7% safety rate on a quadruped. It needed an intermediate physical "Lab" stage of many rollouts to certify its generalization bounds, and its one-step overrides cannot anticipate traps several steps ahead. We include a one-step margin shield without search as a baseline to separate the value of lookahead from the value of calibration.
Why not just clean the sensors with a learned encoder? Compressed representations, as in Prakash et al., remove noise but do not correct dynamics mismatch, and they can discard detail the safety check needs. We run our method with and without such an encoder. The calibration should help in both cases, and if the encoder alone closes the gap, that is informative.
Decisive Checks
The idea is supported if three things hold together. Under shifted, non-Gaussian noise, margin feedback keeps the failure rate within the budget while the fallback fires far less often than under support-based or static filtering. On the race car, zero-shot deployment matches SPiDR's constraint satisfaction with better task progress. And the goal-progress-only reward loses little against the hand-tuned multi-term reward. We also report the gap between the realized miss rate and the failure rate, which directly measures the selection-bias caveat. The idea fails if failure-based adaptation converges just as fast, or if the margin quantile head is so miscalibrated after transfer that the two-sided correction oscillates instead of settling.
References
- ConstrainedZero: Chance-Constrained POMDP Planning Using Learned Probabilistic Failure Surrogates and Adaptive Safety Constraints
- Safe POMDP Online Planning Among Dynamic Agents via Adaptive Conformal Prediction
- Conformal Policy Control
- Deep Local Trajectory Replanning and Control for Robot Navigation
- Hierarchical Apprenticeship Learning with Application to Quadruped Locomotion
- Learning Agile Locomotion via Adversarial Training
- SPiDR: A Simple Approach for Zero-Shot Safety in Sim-to-Real Transfer
- Sim-to-Lab-to-Real: Safe RL with Shielding and Generalization Guarantees