A common way to make a learned robot policy safe is to put a small, fast "safety filter" underneath it. The policy proposes an action. The filter checks whether that action would drive the robot out of a safe region, and if so it nudges the action as little as possible to stay inside. Recent work shows such filters can be learned entirely from logged driving or robot data, with no hand-written physics model and no risky online trial and error. These learned filters share one blind spot: they assume the robot sees everything. In real logs, the dangerous events that matter most are rare. A pedestrian stepping out from behind a parked truck is one example. The logs are also mostly recorded while nothing emerged from the shadows. A filter trained on them will learn that passing a blind corner at speed is safe, because in the data it almost always was.
We propose to learn the safety filter from offline logs after inserting synthetic "phantom" obstacles into the regions each logged frame could not see. The phantoms are placed in the unseen space, move toward the robot at up to a bounded speed, and are erased as soon as the log shows that space to be empty. Their geometry is synthetic, so their collision labels are exact. Only the robot's own braking and turning ability has to be learned from data. The result is a filter that stays as permissive as data-driven barriers in open, visible space and becomes cautious only near the edges of what the robot cannot see. It needs none of the rare dangerous emergence events to be present in the logs.
Core Claim
The claim is specific: offline barrier learning over partially observed states overestimates safety near occlusions. Augmenting the logged transitions with worst-case phantom hazards, confined to the logged unseen regions, fixes this without new robot data. It does so more permissively than hand-built worst-case occlusion rules, because the phantom-relevant margin is learned jointly with the robot's actual stopping ability.
Some terms first. A control barrier function (CBF) is a scalar score $h$ over states, positive when safe. The filter solves a tiny quadratic program (QP), a convex optimization solvable in milliseconds, that keeps $h$ from decaying too fast. V-OCBF supplies the method we reuse. It learns such a barrier purely from offline $(s,a,s')$ logs using expectile regression on a safety Bellman backup, which fits the upper envelope of safe behavior seen in the data without querying unseen actions. It reports a 98.28% safety rate on an autonomous ground vehicle collision-avoidance benchmark, against 95.78% for the offline RL method FISOR and 92.48% for a neural CBF. It assumes full observability, and the research file flags exactly the failure we target. When hazards are hidden in some logged states, the backup cannot tell a clear road from a road masking a pedestrian, and it averages toward "safe."
Mechanism
The barrier is a network $B(x, V)$. Here $x$ is the robot state and $V$ is an egocentric bird's-eye-view grid marking each cell as observed-free, observed-occupied, or unobserved. For each logged frame we compute the unobserved region and seed a phantom set on its frontier. We grow that set at the assumed maximum hidden-agent speed $v_{\max}$ along the logged robot trajectory, and clip it wherever later frames in the same log reveal free space. The instantaneous safety signal is
$$\ell(x_t, V_t) = \min\big(d_{\text{obs}}(x_t),\; d_{\text{ph}}(x_t)\big),$$
where $d_{\text{obs}}$ is the distance to observed obstacles and $d_{\text{ph}}$ is the distance to the current phantom set. The barrier is trained with the V-OCBF backup $B = \min(\ell, \text{expectile}_\tau[B(x', V')])$, where $\tau$ is the expectile level and primes denote the next logged step. The robot's dynamics in the backup come only from the data. The hazard side is analytic.
At deployment the QP treats $V$ as fixed within each control step, since visibility changes with pose and is not smooth. It differentiates $B$ with respect to $x$ only. The backup already encodes how visibility evolves along real trajectories, so this per-step freezing is a controlled approximation rather than an ignored one. We will measure how often it causes one-step violations.
This also gives a concrete handle on a tension the file identifies. The refinement work in Learning a Better Control Barrier Function and Data-Efficient Control Barrier Function Refinement shows that learned corrections expand overly cautious hand-built safe sets. In the first paper, admissible velocity rises from 2.0 to nearly the 3.0 m/s physical limit. Distributionally Robust CVaR-Based Safety Filtering argues the opposite: scarce data should make filters more conservative. In our design, expansion happens where the robot can see. Shrinkage is confined to occlusion frontiers and governed by one interpretable parameter, $v_{\max}$, instead of a global ambiguity radius.
Relation to Prior Work
Two occlusion-aware barrier papers motivate the problem but assume known geometry and no learning. Control Strategies for Pursuit-Evasion Under Occlusion treats the occluded region as an unsafe set in a CBF-QP to keep a target in view. It uses generalized gradients at shadow edges, and it assumes the target's state is known. DiffOcclusion gives smooth distance metrics for convex shapes and a "circulation" constraint that prevents a manipulator from freezing at obstacle boundaries. Its analytic models are exactly what our offline setting lacks. We borrow its circulation constraint as an optional add-on and test whether our filter needs it.
On the perception side, OPV2V, V2VNet and CoBEVT shrink blind spots by sharing features across vehicles. We treat them as complementary: fusion produces a smaller unobserved region $V$, and our filter reasons about whatever remains.
Behavior Planning with Social Perception infers hidden pedestrians from human drivers' reactions. It relies on a noisily-optimal human model and a full probabilistic model-predictive controller.
On the learning side, InterFuser and LogoNav/MBRA show that learned driving and navigation policies still fail on occluded long-tail events. InterFuser loses 15% Driving Score when its rule-based safety controller is removed. Neither provides a formal invariant filter, and our method is designed to sit under policies like these.
Research Program
The work has four parts. First, we build occlusion-augmented variants of existing environments and matching offline datasets. Second, we implement phantom propagation and the phantom-augmented V-OCBF training on top of the published expectile backup. Third, we analyze the conditions under which the learned set is forward invariant against any hidden agent obeying $v_{\max}$. The expected result is a guarantee conditional on barrier fitting error, with the per-step freezing of visibility as the main gap to close. Fourth, we test on simulation and one physical ground robot.
The datasets are logged by mixed-quality behavior policies. They deliberately contain few or no emergence events, to reflect data scarcity. Sizes range from 10k to 1M transitions.
Baselines
The naive reference points are two. One is the nominal planner with no filter. The other is a stop-when-occluded rule that caps speed whenever the forward corridor contains unobserved cells. This rule should be safe and slow; DiffOcclusion documents the freezing this style of conservatism causes. Beating it means matching its hidden-hazard collision rate while making clearly more task progress.
The strongest existing method is V-OCBF, which reports 98.28% safety on its fully observed ground-vehicle benchmark. No reported result was found on Lacuna for V-OCBF or any learned offline barrier under sensor occlusion. We run it on identical logs, with the same network, compute and expectile sweep over $\tau \in \{0.9, 0.99\}$, in two versions: with raw observations, and with the same visibility grid $V$ as input but no phantoms. The second version is the key ablation. It isolates whether phantom augmentation, not merely access to visibility information, drives any gain.
A composed baseline takes the minimum of this V-OCBF barrier and an analytic worst-case phantom barrier. The analytic barrier uses a hand-specified braking model at nominal deceleration, tuned with the same budget.
We also compare against the DR-CVaR filter, fed phantom emergence samples as predictions. As an online reference we include PPO-Lagrangian from Safety-Gymnasium, which reports a 98% violation reduction at a 45% reward cost on velocity tasks. It is given environment interaction the offline methods do not get.
Beating the field means two things. First, a lower hidden-hazard collision rate than both V-OCBF variants at matched data. Second, higher progress than the stop-when-occluded rule and the composed barrier at equal or lower collision rate, with confidence intervals over at least five seeds.
Why not just…?
Why not just give V-OCBF the visibility map and let it learn? The logs rarely contain a hazard actually emerging, so the backup never sees the unobserved cells associated with failure. It will learn to ignore them. This is the Gap 2 argument in the file, and the visibility-input ablation above tests it directly. If that ablation matches our method, the phantom idea is unnecessary and we will report so.
Why not just take the minimum of a learned barrier and a hand-written occlusion barrier? This is the most serious threat. The hand-written term needs a braking model, which the offline setting does not provide. Guessing that model either wastes speed if the guess is conservative or causes collisions if it is optimistic. A pointwise minimum of two barriers also does not guarantee the QP stays feasible, and the DR-CVaR paper reports that feasibility is exactly what breaks in tight spaces. We expect the gap to be largest on platforms whose braking differs from the assumed model, such as the Car and Racecar agents or a loaded ground robot. If the composed filter matches ours even there, the practical contribution shrinks to the augmentation recipe.
Why not just remove the occlusion with vehicle-to-vehicle fusion? Fusion helps a lot. On OPV2V, detection [email protected] rises from 0.406 to 0.734 in the default towns. That still leaves misses, needs connected partners, and degrades under latency and packet loss. We run our filter on fused visibility maps as well, to show that the two approaches stack rather than compete.
Why not just brake whenever something is occluded? It is safe but freezes the robot. Beating it on progress at matched safety is our naive-baseline criterion.
Proposed Experiments
In Safety-Gymnasium, we add opaque pillars and walls to the Point, Car and Racecar navigation tasks. Hazards spawn hidden and emerge at random speeds up to a true $v^\star$. We report hidden-hazard collision rate, constraint cost, reward, fraction of time stopped, QP infeasibility rate and solve latency. We also sweep dataset size to test whether phantom augmentation lets small logs reach the safety of large ones. Separately, we test the filter's sensitivity to setting $v_{\max}$ above and below the true $v^\star$.
In CARLA 0.9.10.1, we construct occluded-pedestrian scenarios behind parked vehicles, the setting InterFuser highlights. We place our filter under a pretrained nominal driving policy and report collision and infraction rates and route completion. InterFuser's reported leaderboard Driving Score of 76.18 is context only, since our scenario suite differs.
On a Clearpath Jackal in a pillar environment like that of the pursuit-evasion study, a person steps out from behind pillars. We compare collisions and traverse time for our filter, V-OCBF, and the stop-when-occluded rule.
The decisive check has two conditions. In the regime where emergence events are nearly absent from the logs, our filter must cut hidden-hazard collisions relative to the visibility-input V-OCBF ablation. It must also beat the composed analytic barrier on progress at a braking model mismatch. If either fails, the central claim is wrong.
References
- V-OCBF: Learning Safety Filters from Offline Data via Value-Guided Offline Control Barrier Functions
- Learning a Better Control Barrier Function
- Data-Efficient Control Barrier Function Refinement
- Distributionally Robust CVaR-Based Safety Filtering for Motion Planning in Uncertain Environments
- Control Strategies for Pursuit-Evasion Under Occlusion Using Visibility and Safety Barrier Functions
- DiffOcclusion: Differentiable Optimization Based Control Barrier Functions for Occlusion-Free Visual Servoing
- OPV2V: An Open Benchmark Dataset and Fusion Pipeline for Perception with Vehicle-to-Vehicle Communication
- V2VNet: Vehicle-to-Vehicle Communication for Joint Perception and Prediction
- CoBEVT: Cooperative Bird’s Eye View Semantic Segmentation with Sparse Transformers
- Behavior Planning of Autonomous Cars with Social Perception
- Safety-Enhanced Autonomous Driving Using Interpretable Sensor Fusion Transformer
- Learning to Drive Anywhere with Model-Based Reannotation
- Safety Gymnasium: A Unified Safe Reinforcement Learning Benchmark