Radar is useful in self-driving cars mainly because it keeps working at night, in rain and in tunnels, where cameras and LiDAR degrade. Radar data is too noisy and abstract for people to label, so it is usually trained by cross-modal supervision. A camera or LiDAR "teacher" model runs on synchronized recordings and its outputs become training labels for a radar-only "student." This creates a paradox. The teacher is reliable in exactly the conditions where radar is least needed and unreliable in exactly the conditions where radar matters most. Current pipelines either use teacher labels everywhere, absorbing the teacher's night-time misses, or quietly rely on clean recordings, so the student never learns from the adverse data it will face.
We propose to learn a small reliability model that looks at the radar evidence and the teacher's features together and predicts, for each location in the bird's-eye-view grid, whether the teacher's label there can be trusted. That prediction then weights the distillation loss. The reliability model is trained without any human labels. On clear-day recordings we artificially degrade the teacher's inputs (darkening, noise, glare, LiDAR point dropout) and teach the model to predict where the degraded teacher disagrees with the clean teacher. The model sees the radar and the degraded teacher, never the clean one. We then apply it to real night and rain recordings. The central claim is that the student's own sensor contains the information needed to detect the teacher's errors, especially its missed objects, and that using it lets a radar student benefit from adverse-condition data instead of being harmed by it.
Core Claim and What Is New
The domain shift in this setting falls on the teacher, not the student. Prior work treats cross-modality domain shift as something that hits the deployed model. AdaPose adapts a WiFi pose student across rooms, and SIFA and FAN-Net adapt segmenters across scanners. Radar distillation has the inverse structure. LiCROcc reports qualitatively that LiDAR models degrade on rainy nights while radar models stay stable. That observation motivates this proposal: the label source becomes unreliable precisely where the student is still informative.
Teacher confidence cannot catch the dominant failure. A teacher blinded by darkness tends to output confident "background" over a pedestrian it cannot see. The radar may still show a strong return with nonzero Doppler (radial velocity) at that spot. Only a model that compares the two sensors can flag the miss.
The closest existing work is Fusion-then-Distillation (FtD++). It also fights "negative mimicking" from a degraded modality. Its debiased pseudo-labeling down-weights samples where camera and LiDAR predictions disagree, and it reports +11.8 mIoU over source-only and +9.4 over xMUDA on nuScenes Day→Night 3D segmentation. Our proposal differs from FtD++ in three ways:
- FtD++ works with dense LiDAR and RGB, both of which produce semantic predictions. We work with a radar student whose early predictions are too poor to act as a voter.
- FtD++'s reliability signal is a fixed variance heuristic. Ours is learned, with explicit supervision from controlled degradations.
- We distinguish teacher false negatives from false positives instead of treating disagreement symmetrically.
CORENet, CRKD and LiCROcc supply the distillation machinery we reuse: LiDAR-derived radar targets, and residual, relation and response distillation in bird's-eye view. None of them models when the teacher is wrong. CORENet's own discussion notes that its LiDAR-proximity masks become corrupted when LiDAR degrades in rain.
Mechanism
The reliability model uses cross-modal attention. In this mechanism, features from one sensor act as "queries" that look up and weight features from the other sensor ("keys" and "values"). Here each radar grid cell queries the teacher's feature map within a local window. Local attention keeps the cost linear in grid size, which addresses the quadratic-cost concern raised for dense grids. Attention lets a radar cell find the matching teacher evidence despite radar's coarse angular resolution and small calibration offsets. A fixed same-cell comparison would be confused by those offsets.
We add a learned gate in the style of gated cross-attention, which mixes attended context with the original features to prevent one modality from overwriting the other. The gate keeps native radar signatures such as Doppler intact in the reliability features. The head outputs two per-cell probabilities: $p^{\text{miss}}_c$ that the teacher missed an object at cell $c$, and $p^{\text{fake}}_c$ that the teacher hallucinated one.
The student is trained with
$$ \mathcal{L} = \sum_c w_c \,\ell\big(s_c, t_c\big), \qquad w_c = \mathrm{sg}\big[\,1 - \max(p^{\text{miss}}_c, p^{\text{fake}}_c)\,\big], $$
where $s_c$ and $t_c$ are student and teacher outputs at cell $c$, $\ell$ is the per-cell distillation loss of whichever base pipeline we use, and $\mathrm{sg}$ stops gradients. The stop-gradient matters. The reliability head is trained only by the synthetic-degradation objective, never by the distillation loss, so it cannot learn the trivial solution of zeroing every weight.
Cells flagged as teacher misses are excluded from the negative (background) loss rather than relabeled as positives. Relabeling would let reliability errors inject false objects. We keep this conservative choice unless the experiments show that relabeling helps.
Research Program
The work has three stages.
Stage 1: degradation library and reliability head. We build a library of input degradations: camera low-light with sensor noise, glare, motion blur, and range-dependent LiDAR dropout with spurious near-range returns. On clear-day frames we compute clean-teacher and degraded-teacher outputs. We define miss and fake targets from their disagreement. We then train the reliability head from radar plus degraded-teacher features.
Stage 2: validation against real teacher errors. Before any student training, we test whether the head generalizes from synthetic to real degradation. On real night, rain and tunnel frames, we mark the teacher's errors against human annotations and measure how well the head ranks them, using area under the ROC curve. We compare it to teacher entropy and to a hand-written radar rule. If the head does not beat teacher entropy here, the project's premise has failed. Stage 2 is designed so this is visible early and cheaply.
Stage 3: student training. We train radar students with the weighted loss on all conditions, using no human labels. We evaluate them against human labels separately for each condition.
Baselines
The naive reference point is a radar student distilled only from clear-day frames, which discards adverse recordings entirely. It is the no-risk choice a practitioner might make. Beating it on night and rain shows that adverse unlabeled data has value once the teacher's errors are handled.
The second reference is the same student distilled from all frames with the teacher's raw labels, the implicit practice in CORENet. Comparing these two baselines also measures how much raw adverse supervision helps or hurts. That gap determines how much room the proposal has.
The third baseline is teacher-confidence weighting: per-cell weights from teacher entropy, with the threshold tuned on a held-out clear-day split. It is the cheapest form of reliability estimation.
The strongest existing method is FtD++-style debiased pseudo-labeling (FtD++). We reimplement it with weights from the prediction variance between the camera branch and the LiDAR branch of a camera-LiDAR teacher, plugged into the same student and loss. Its reported figure is +9.4 mIoU over xMUDA on nuScenes Day→Night 3D segmentation. That is a camera-LiDAR segmentation number, and we found no reported result on Lacuna for any teacher-reliability method applied to radar students.
We also run a hand-crafted radar veto. It ignores the background loss wherever radar shows a high-intensity return with nonzero Doppler, and it ignores the foreground loss wherever a teacher box contains no radar return within a tolerance. This is the most dangerous simple competitor, and it isolates whether learning adds anything beyond physics rules.
For fairness, all methods share the same student architecture, the same teacher checkpoints, the same unlabeled training frames, the same number of training steps, and an equal hyperparameter search budget. That budget is a fixed number of trials, tuned on clear-day validation only, because human labels in adverse conditions are reserved for testing.
Ablations isolate the key ingredients. We replace attention with concatenation followed by convolutions at matched parameter count. We remove the gate. We remove radar input from the reliability head, which makes it a learned teacher-only confidence model. That last ablation directly tests the claim that the student's sensor carries the error signal.
An oracle student trained on human labels in all conditions gives the upper bound. Beating the strongest baseline means a consistent gain in night, rain and tunnel average precision across three seeds, with clear-day performance unchanged within seed noise.
Why not just…?
Why not just threshold on teacher confidence? Confidence detects uncertain predictions, but a teacher blinded by darkness is usually confidently wrong about empty space. Stage 2 tests this directly by measuring miss-detection AUROC for entropy versus the reliability head on real night frames. If entropy matches the head, the simpler method wins and we will say so. The no-radar ablation also bounds how much of any gain comes from learned confidence alone.
Why not just use the hand-written radar veto? This is the real threat. Doppler and return intensity are strong cues, and Multi-View Radar Semantic Segmentation and CARRADA show how informative the Doppler view is: 52.9% versus 32.6% mIoU for range-Doppler versus range-angle with FCN-8s. Where we expect the rule to fall short is static obstacles, which have zero Doppler, and multipath ghosts, which have strong returns but no object. The learned head can use context such as road layout in the teacher's features to handle these. We will report results split by moving versus static objects. If the veto matches the learned head on both, the contribution reduces to the evaluation protocol and the veto itself.
Why not just use FtD++'s variance weighting? It needs two semantic predictors that fail independently. At night the camera fails but LiDAR does not, so variance flags camera errors correctly. In rain both degrade, and agreement between two wrong predictors produces low variance. Our condition-by-condition results on rain and tunnel frames will show whether this matters. If the advantage appears only at night, the honest conclusion is that variance weighting suffices except when all teacher sensors share the failure.
Why not just self-train the student on its own predictions? Self-training refines a model whose errors are already correlated with its training labels, and early radar students are weak. LiCROcc's radar-only student reaches only 8.5 mIoU on nuScenes-Occupancy. We include one round of confidence-filtered self-training as an add-on to every method, so it is tested as a complement rather than ignored.
Proposed Experiments
Primary testbed: Dual-Radar. This is the dataset used by CORENet. It pairs 4D radar with LiDAR across daytime, dark tunnels, rain and night, and has human 3D box labels; CORENet reports Car 3D AP of 49.46% and BEV AP of 54.38% with CasA-V. The file gives no per-condition breakdown, so no reported result was found on Lacuna for condition-split radar AP on this dataset, and we will establish it. We use two teachers: LiDAR-only, which degrades in rain, and camera-only, which degrades at night and in tunnels. Using both tests whether the reliability head handles different teacher failure modes. We report Car, Pedestrian and Cyclist 3D and BEV AP per condition and per range bin. Pedestrians matter because CORENet reports weakness on small non-rigid objects, which are also where teacher misses are most likely.
Second testbed: nuScenes-Occupancy. We use the LiCROcc setup, whose radar-only student reaches 8.5 mIoU. We swap our weighting into its distillation losses and evaluate on rain and night subsets. Two outcomes would carry over the fewest assumptions from the primary testbed: whether radar-only mIoU on rainy-night scenes improves, and whether the 30–50 m collapse LiCROcc reports changes.
Control: CARRADA. CARRADA is a clean test track where the camera teacher is reliable. The method should change nothing here, and a drop would indicate the head is suppressing valid labels.
Decisive checks. First, Stage 2 must show the head ranking real teacher errors above entropy and above the no-radar ablation. Second, the weighted student must beat both raw-label distillation and clear-day-only distillation on adverse conditions, without losing clear-day performance. Third, it must beat FtD++-style variance weighting and the radar veto on at least rain or static-object subsets. If the first check fails, the synthetic-to-real transfer of teacher degradation does not hold and the project stops there. If only the third fails, the paper becomes a careful negative result showing that simple physics vetoes are enough.
References
- AdaPose: Toward Cross-Site Device-Free Human Pose Estimation With Commodity WiFi
- Synergistic Image and Feature Adaptation: Towards Cross-Modality Domain Adaptation for Medical Image Segmentation
- Fan-Net: Fourier-Based Adaptive Normalization for Cross-Domain Stroke Lesion Segmentation
- LiCROcc: Teach Radar for Accurate Semantic Occupancy Prediction Using LiDAR and Camera
- Fusion-Then-Distillation: Toward Cross-Modal Positive Distillation for Domain Adaptive 3D Semantic Segmentation
- CORENet: Cross-Modal 4D Radar Denoising Network with LiDAR Supervision for Autonomous Driving
- CRKD: Enhanced Camera-Radar Object Detection with Cross-Modality Knowledge Distillation
- An Interpretable Framework for Drug-Target Interaction with Gated Cross Attention
- Multi-View Radar Semantic Segmentation
- CARRADA Dataset: Camera and Automotive Radar with Range- Angle- Doppler Annotations