Robots and cars that learn from demonstrations increasingly use diffusion policies. A diffusion policy is a generator that starts from random noise and refines it, step by step, into an action. Its main appeal is that it can represent several distinct good actions, such as passing an obstacle on the left or on the right, without blending them into a bad average. This note argues that the same property becomes a hazard when the robot's sensors are blocked. Suppose demonstrations were collected by someone who could see past the occluder. The diffusion policy then learns "go" in the episodes where the hidden road was clear and "yield" in the episodes where it was not. At test time, unable to see, it does not hedge. It draws "go" with roughly the probability that the road happened to be clear in the training data. The policy's ability to represent several actions has turned uncertainty about the world into a coin flip over actions.
We propose to separate the two kinds of spread. The diffusion policy still proposes a small set of candidate actions. Each candidate is then scored by a critic, a network that predicts future reward. This critic was trained cheaply in simulation with full visibility, and at decision time it is averaged over several sampled guesses of what lies in the occluded region, using a risk-averse average. The same separation fixes a sample-efficiency problem during reinforcement-learning fine-tuning. Exploration bonuses that reward critic uncertainty currently cannot tell "the critic hasn't learned this yet" from "the critic can't know what's behind the truck," so they waste interactions chasing the second. Our bonus counts only the first.
Core Claim
Under occlusion, a multi-modal policy needs an explicit outer layer that keeps two sources of spread apart. One is the spread the agent may choose among: strategy modes such as left or right. The other is the spread it must hedge against: hypotheses about the hidden state. Providing this layer yields two measurable gains. First, it gives safer and higher-success behavior than either a unimodal Gaussian policy (which hedges by accident, through averaging) or a raw diffusion policy (which gambles). Second, it reaches a target success rate in fewer environment interactions than uncertainty-driven diffusion fine-tuning that lumps both spreads together.
Concretely, let $o$ be the occluded observation and $z_1,\dots,z_M$ be samples from a generator $p(z \mid o)$ of the hidden region, for example an occupancy grid behind the occluder. Let $a_1,\dots,a_N$ be candidate actions drawn from the diffusion policy, and let $Q_k(o, z, a)$ for $k=1..K$ be an ensemble of critics trained with privileged access to the true hidden state. Candidate $a$ is scored as
$$S(a) = \mathrm{CVaR}_\alpha\big\{\bar Q(o, z_m, a)\big\}_{m=1}^{M} + \lambda\, \mathrm{std}_k\Big[\tfrac{1}{M}\textstyle\sum_m Q_k(o, z_m, a)\Big],$$
where $\bar Q$ is the ensemble mean, $\mathrm{CVaR}_\alpha$ is the mean of the worst $\alpha$ fraction of hypotheses, and $\lambda$ weights exploration. The first term hedges against the hidden state. The second term is an exploration bonus computed only from disagreement among critics after averaging over hypotheses, so spread caused purely by occlusion earns no bonus. During training $\lambda>0$; at deployment $\lambda=0$.
Relation to Prior Work
Three papers supply the method. The candidate-then-score structure comes from DF-ExpEnse. It samples candidates from a pretrained diffusion policy and ranks them by the minimum of a critic ensemble plus ensemble standard deviation, which makes fine-tuning sample-efficient. That method assumes the robot can see everything it needs, so its uncertainty bonus mixes ignorance about task dynamics with occlusion-induced noise. Our proposal changes exactly that scoring rule. Diffusion Policy supplies the policy class and the argument that the starting noise forces the generator to commit to one mode. That commitment is what we diagnose as gambling under occlusion. DPPO supplies the strongest on-policy fine-tuning alternative. It keeps exploration near the demonstrations, but it was never tested with sensors blocked.
Two papers motivate the hypothesis generator but also constrain it. DiffCP and the Occlusion-Aware Diffusion Model reconstruct missing scene features or pedestrian tracks with diffusion. Both stop at perception and both acknowledge hallucination risk. DiffCP also reports about 100ms reconstruction time, which rules out a feature-level diffusion transformer inside a control loop. We therefore use the generator to produce a set of hypotheses scored by a risk-averse average, not a single filled-in scene, and we use a light occupancy-grid generator. People as Sensors is the closest end-to-end precedent: a variational autoencoder infers occluded occupancy and feeds a Gaussian PPO planner, raising crowd-navigation success from 79% to 91%. It collapses inference into one latent passed to a unimodal policy, which is the design we contrast against.
Two driving papers frame the evaluation. Rethinking Closed-loop Training for Autonomous Driving shows standard RL is sample-hungry and collision-prone in occluded and cut-in scenarios. Its TRAVL method scores trajectory candidates with a learned value and imagined rollouts, but it picks deterministically, has no generative proposal distribution, and has no explicit treatment of hidden-state hypotheses. Shared Information-Based Safe Behavior Planning shows that a safety projection layer removes unsafe actions in CARLA. This is a competitor we must beat on throughput, not just on safety.
Research Program
The first stage is a diagnosis, deliberately small. We build an occluded crossing in CARLA and an analogous manipulation task: a RoboMimic-style pick in which the target's position is hidden behind an occluder in the camera view on some episodes. In both, demonstrators act with privileged visibility. We train a diffusion policy and a Gaussian policy by behavior cloning on the occluded observations only. We then measure, as a function of the fraction of demonstrations in which the hidden region was benign, how often each policy commits to the aggressive action while occluded. The prediction is that the diffusion policy's commit rate tracks the benign fraction, while the Gaussian policy outputs an intermediate "creep" action. If the diffusion policy does not gamble, the motivation fails and we stop here.
The second stage builds the components. The hypothesis generator is a conditional generator over occupancy in the occluded region, trained on simulator ground truth. We start with a variational autoencoder in the style of People as Sensors and treat the generator architecture as an ablation. The critic ensemble is trained with privileged inputs, meaning it sees the true hidden state $z$ during training. This makes value learning a fully observed problem, which is where off-policy reuse is most stable. At decision time the true state is replaced by hypotheses. The diffusion policy itself is frozen during early fine-tuning, as in DF-ExpEnse. It is then updated by refitting it to the actions the selector chose, so it gradually stops proposing gambles.
The third stage tests calibration and the regime where hypotheses should collapse. We check how often the true hidden state falls inside the generated hypothesis set; the risk-averse average is only meaningful if the generator covers real hazards. In a cooperative variant built on CARLA/OpenCDA, as used by OPV2V, a second vehicle shares an occupancy estimate of the blind spot. Here the hypothesis spread should shrink and the selector should automatically become less cautious. That would be evidence the method is responding to information rather than simply being conservative.
Baselines
The naive reference is the behavior-cloned diffusion policy executed directly, with no selector and no fine-tuning. Its commit-under-occlusion rate and collision rate set the floor, and the diagnosis stage characterizes it. Beside it we run a behavior-cloned Gaussian policy fine-tuned with PPO. In the cited driving study, Gaussian PPO reached collision rates "as high as 24.9%" on the Targeted benchmark (Rethinking Closed-loop Training). Under occlusion its accidental averaging may make it look cautious, so we report both pass rate and time to clear the intersection.
The strongest competitor on the sample-efficiency question is DF-ExpEnse (DF-ExpEnse), run with critics that see only observations. To make it a fair test of whether our decomposition matters, rather than of whether it has memory, we give it the same frame-history encoder. We run it with the same candidate count $N$, ensemble size $K$ (its ablations favor around 10), environment-step budget, and hyperparameter search budget. The file reports that it outperformed DSRL and ResFiT on Tool Hang but gives no numerical success rate: no reported result found on Lacuna for DF-ExpEnse success percentages. We therefore re-run it ourselves on all tasks. Beating it means reaching a fixed target success rate, defined as 90% of DF-ExpEnse's own final success, in fewer environment steps, and finishing with a lower collision rate. Each comparison uses at least five seeds.
DPPO (DPPO) is the on-policy reference. It reports above 90% success on the bimanual Transport task from state and 80% zero-shot real-robot success on Furniture-Bench One-leg, but has no occluded result. We run it with a privileged value function, the standard asymmetric actor-critic trick, so that privileged information is available to both methods.
On driving, TRAVL is the reference for the Targeted scenario family. It reports an 86.5% pass rate and a 2.6% collision rate, with Rainbow at a 4.8% collision rate (Rethinking Closed-loop Training). Our occluded variants differ from that benchmark, so we re-implement TRAVL-style candidate scoring on our scenarios rather than compare to the published number directly. We also add the safe action projection of Shared Information-Based Safe Behavior Planning on top of Gaussian RL.
The ablations isolate the key ingredient. One replaces the risk-averse average with a plain mean over hypotheses. Another replaces the hypothesis generator with "assume the hidden region is empty" and, separately, with "assume worst case." A third restores the lumped uncertainty bonus (standard deviation across all critic and hypothesis pairs) while keeping everything else fixed. That last ablation is the cleanest test of the sample-efficiency claim.
Why not just…?
Why not just use a Gaussian policy, since averaging is effectively hedging? Under occlusion the Gaussian's mean action may indeed be a reasonable creep, and we take this seriously. The problem is that the same averaging is catastrophic for genuine strategy modes, such as left versus right around an obstacle, as Distributional Policy Optimization and Pearce et al. document. A Gaussian cannot hedge over the hidden state while committing over strategy. Our scenarios combine both kinds of spread, an occluded crossing plus a lane-level obstacle, so the experiment will show whether the Gaussian's accidental caution survives. If Gaussian PPO matches us on scenarios that have only occlusion and no strategy modes, we will report that.
Why not just give DF-ExpEnse or DPPO memory and a privileged critic during training? This is the most serious threat. Frame history does not reveal what was never seen. A privileged critic used only during training improves value targets, but at decision time the critic still conditions on the observation alone, so occlusion spread re-enters both selection and the exploration bonus. Our method is, in effect, the deployable form of a privileged critic: averaging the critic over generated hypotheses approximates its expectation over the hidden state. The lumped-bonus ablation and the DPPO-with-privileged-value baseline settle the question. If they reach the target success rate in as few steps as we do, the decomposition adds nothing, and we would say so.
Why not just put a safety shield on any policy? The shield in Shared Information-Based Safe Behavior Planning eliminated unsafe actions, but it acts by projection or emergency stop. At blind corners this tends to produce freezing. We compare time to clear, throughput, and pass rate at matched collision rates. A shield that matches our collision rate but halves throughput does not answer the question.
Why not just reconstruct the hidden region once and act on the reconstruction? Single reconstructions are exactly where the hallucination risk flagged by DiffCP and the Occlusion-Aware Diffusion Model bites: a confident "clear" guess produces a confident "go." The "assume empty," plain-mean, and single-sample (M=1) ablations measure this directly.
Decisive Checks
Three results would establish the claim. First, the diagnosis must show that the behavior-cloned diffusion policy's commit-under-occlusion rate tracks the benign fraction in the demonstrations. Second, on occluded CARLA and the occluded manipulation task, the full method must reach the target success rate in fewer environment steps than DF-ExpEnse with history, and fewer than the lumped-bonus ablation, while matching or lowering collisions. Third, in the cooperative variant, adding a collaborator's view must reduce hypothesis spread and increase speed through the intersection without increasing collisions.
Two results would refute the claim: the lumped-bonus ablation matching the full method on sample efficiency, or the generator's hypothesis set covering the true hidden state too rarely for the risk-averse average to matter. Latency is reported throughout. The selector costs $N \times M$ critic evaluations plus one generator pass, and the run counts only if it fits inside the CARLA control period.
References
- DF-ExpEnse: Diffusion Filtered Exploration for Sample Efficient Finetuning
- Diffusion Policy: Visuomotor Policy Learning via Action Diffusion
- Diffusion Policy Policy Optimization
- DiffCP: Ultra-Low Bit Collaborative Perception via Diffusion Model
- Occlusion-Aware Diffusion Model for Pedestrian Intention Prediction
- Occlusion-Aware Crowd Navigation Using People as Sensors
- Rethinking Closed-Loop Training for Autonomous Driving
- Shared Information-Based Safe And Efficient Behavior Planning For Connected Autonomous Vehicles
- OPV2V: An Open Benchmark Dataset and Fusion Pipeline for Perception with Vehicle-to-Vehicle Communication
- Distributional Policy Optimization: An Alternative Approach for Continuous Control
- Imitating Human Behaviour with Diffusion Models