Stopping Radar Detectors from Inheriting Their Camera Teacher's Blind Spots

Automotive radar is valuable because it keeps working in darkness, glare and fog, where cameras fail. Raw radar heatmaps are almost impossible for people to label, so the standard recipe is to let a camera do the labeling. A camera detector finds objects in daylight-quality images, its detections are projected into the radar's bird's-eye view, and a radar-only network learns to reproduce them. This creates an awkward circularity. The camera teacher is weakest in exactly the conditions that motivate radar, so the radar student is trained on labels that go missing at night and in glare. If the student can tell from the radar signal alone that "this is the kind of scene where labels go missing," it will learn to stay silent there too, and the camera's blind spots get copied into a sensor that physically does not have them.

This note proposes to measure and remove that inheritance with adversarial debiasing. Adversarial debiasing is a training method in which a second network, the adversary, tries to predict a nuisance attribute from the model's internal features, and the main model is penalized whenever it succeeds. The nuisance attribute here is the camera's imaging condition, measured automatically from the image (exposure, luminance, glare), so no human labels are needed. The choice of attribute rests on a simple physical argument. Illumination has no effect on radar returns at 77 GHz, so any information about lighting that a radar encoder carries must come from scene correlations, and the most dangerous such correlation is the teacher's condition-dependent failure. For this attribute, enforcing invariance should cost almost no true task signal. That property is rare in debiasing, where removing a protected attribute usually also removes useful information.

Core Claim

The claim has three parts. First, a radar student trained with camera supervision inherits the camera teacher's failures only to the extent that its features can predict the teacher's imaging condition. Second, this predictability is measurable with a simple probe on frozen features. Third, adversarially removing the information that predicts imaging condition restores the student's recall in conditions where the camera fails, without hurting daytime accuracy. This would turn a diagnostic finding (the radar model "knows" it is night) into an intervention applied during training. It would also give a way to separate two cases: inheritance through recognizable scene context, which adversarial training can fix, and inheritance through misses on specific objects, which it cannot and which needs label correction instead.

Relation to Prior Work

The camera-to-radar supervision pipeline itself comes from the CRUW and RODNet line of work. Rethinking of Radar's Role projects Mask R-CNN detections into radar coordinates through a ground plane re-estimated every frame. RODNet and its severe-conditions variant then train a 3D convolutional student to predict confidence maps, which are images showing where objects are likely to be, from range-azimuth radar heatmaps. These papers motivate the proposal, because they justify radar by camera failure while relying on the camera for labels. They also supply the student architecture we reuse. None of them measures whether the student copies the teacher's condition-dependent misses.

Radical pre-trains radar encoders with physics-consistent augmentations (dropping virtual antennas, adding phase noise) and aligns radar features with frozen CLIP features from paired camera frames. This aligns radar with the camera even more tightly, so it is a likely carrier of the bias we target rather than a remedy for it. We use it as a strong pre-trained starting point.

Two works supply the method or the diagnostic idea. MABR attaches adversaries with gradient reversal to several encoder layers. A gradient reversal layer passes features forward unchanged and flips the sign of the adversary's gradient on the way back, so the encoder learns to defeat the adversary. MABR also reports that lower layers resist debiasing. We borrow the multi-layer placement, but we use an attribute that is measured physically rather than discovered blindly. Invisible Shortcuts shows that vision encoders encode the camera hardware that captured an image. Our probe step asks the analogous question for radar encoders and imaging condition. BCG-DEC measures how much a radar classifier relies on human-defined concepts using concept activation vectors, but it does not change the model. We use its style of probing only as a diagnostic.

KANFace finds that high-dimensional embeddings retain bias that resists scrubbing. This is a warning for us: adversarial training can leave a feature space looking clean to the adversary while information remains. We therefore evaluate with fresh probes trained after the fact, not with the training adversary's accuracy.

Research Program

The work proceeds in three stages. The first stage is diagnosis. We train a standard RODNet student on CRUW with camera-generated confidence maps. We freeze it and fit small probes on several intermediate layers to predict a per-frame imaging-condition score computed from the camera image: mean luminance, fraction of saturated pixels, and a binned day/dusk/night label. A control probe trained on shuffled condition labels sets the chance level. If the probes cannot recover condition, the inheritance hypothesis is wrong for this data and the project stops here. That outcome would itself be a useful negative result.

The second stage is the intervention. We add gradient-reversal adversaries that predict the condition score at the layers where the probes succeeded, and we sweep the adversary weight. The training objective is

$$\min_{\theta,\phi}\max_{\psi}\; \mathcal{L}_{\text{conf}}(C_\phi(E_\theta(x)), \hat{y}) - \lambda\, \mathcal{L}_{\text{cond}}(D_\psi(E_\theta(x)), c).$$

Here $x$ is the radar heatmap sequence, $\hat{y}$ is the camera-derived confidence map, $c$ is the image-derived condition score, $E_\theta$ is the radar encoder, $C_\phi$ is the confidence-map head, $D_\psi$ is the condition adversary, and $\lambda$ trades detection loss against invariance. We deliberately exclude rain and fog from $c$. Those conditions really do attenuate radar, so invariance to them would erase true signal.

The third stage asks which kind of inheritance the intervention can fix. We need ground truth that is clean in the degraded conditions, so we build it by construction. We take daytime CRUW sequences, where the teacher is reliable, and keep their original teacher labels as ground truth. We then darken the camera images, or add synthetic glare, before rerunning the teacher on a subset of sequences. We make that subset correlated with a particular scene type, for example parking lots. In one variant the degraded subset is correlated with scene type, which models context-driven inheritance. In another it is uncorrelated with scene type and the injected misses fall only on low-contrast objects, which models object-driven inheritance. The prediction is that adversarial training recovers most of the lost recall in the first variant and little in the second. A result in which it "helps" in both variants would suggest it is acting as a generic regularizer rather than through the proposed mechanism.

Baselines

The naive reference point is the RODNet student trained directly on camera-generated confidence maps with no correction. This is the published pipeline. Its strongest reported numbers are 86% average precision and 88% average recall on CRUW in RODNet (JSTSP), and 83.76% AP and 85.62% AR in the severe-conditions version. Rethinking of Radar's Role also reports up to 87.8% DQF1, a score that combines localization closeness with classification confidence. We retrain this baseline ourselves with identical splits so that its numbers are directly comparable. The published figures serve as a sanity check that our reimplementation is faithful.

The strongest practical competitor is condition filtering. This baseline drops, or down-weights in proportion to the condition score, every frame whose camera image falls below a luminance or glare threshold, and then trains the same student. It is strong because it removes the corrupted labels directly with no minimax game. We tune its threshold on the same validation budget as $\lambda$.

A second competitor weights each label by the teacher's own detection confidence. This is a common noisy-label heuristic and needs no condition measurement at all.

A third competitor initializes from Radical-style pre-training. Radical reports a 5.8% mAP gain over supervised training on Radatron and an 11.5% gain at 1% labels. No reported result was found on Lacuna for Radical on CRUW, so we run it ourselves. This tests whether better radar features already resist inheritance. We also apply the adversary on top of it.

Two ablations isolate the key ingredient. In the first, the adversary predicts a shuffled condition label, which controls for generic regularization from gradient reversal. In the second, the adversary predicts scene type instead of illumination, which tests whether the benefit comes from the physically motivated attribute specifically. All methods use the same architecture, epochs, data and number of tuning trials.

Beating the baselines means the following. The method must reduce the inheritance gap, defined as the student's recall on degraded-condition frames measured against clean ground truth relative to its recall on clean frames, by more than filtering does, at matched daytime AP within one point. On real CRUW night data, it must improve night-stratified recall over filtering.

Why not just…?

Why not just drop the frames where the camera is unreliable? This is the serious threat. If teacher degradation were binary and easy to detect, filtering would probably match the proposal. The proposal has two advantages in principle. Real degradation is graded and partial, as at dusk or with glare in part of the frame, so a threshold either keeps bad labels or throws away much of the data. And dropping night frames removes the night scene types from training altogether, so a student can still associate those contexts with low confidence when it encounters them at test time. Neither advantage is guaranteed. The bias-injection sweep varies how much of the data is degraded and how strongly degradation correlates with scene type, and it reports where, if anywhere, adversarial training wins. If filtering ties everywhere, we will report that the simple fix is sufficient.

Why not just treat it as label noise and use teacher confidence weighting? Confidence weighting down-weights uncertain detections, but it cannot help with missed objects. Where the camera saw nothing there is no detection to down-weight, so the missing label is a confident "background" label. The object-driven injection variant shows directly how much this baseline recovers. We expect little recovery, because the injected misses carry no low-confidence trace.

Why not just use better radar pre-training such as Radical? Radical's alignment with CLIP ties radar features more closely to camera semantics, which could make inheritance worse rather than better. Its reported gains come from Radatron vehicle detection and say nothing about recall split by imaging condition. Running the diagnostic probe and the inheritance gap on a Radical-initialized student answers this empirically. If pre-trained features already carry no condition information, the adversary should be unnecessary, and the probe will show it.

Proposed Experiments

The decisive checks run in order. First, the probe must recover imaging condition from RODNet features well above the shuffled-label control. Second, in the context-correlated injection setting, the adversary must close substantially more of the inheritance gap than filtering and confidence weighting, while the shuffled-label adversary closes almost none. Third, in the object-driven injection setting, the adversary should give little benefit, which confirms the mechanism rather than a regularization effect. Fourth, after training, freshly trained probes, not the training adversary, must fail to recover condition. This guards against the hidden retention that KANFace warns about. Probes at every layer will show whether the lower-layer resistance reported by MABR also occurs in radar.

On real data, we report AP and AR under the object location similarity metric on CRUW, stratified into day, dusk and night/glare bins. For the night/glare bin we use a small human-verified subset. Annotators verify camera-projected labels against offline temporal tracking from adjacent well-lit frames, rather than labeling raw radar heatmaps. As a transfer check, we repeat the probe-then-adversary procedure on CARRADA's range-Doppler segmentation, where CARRADA reports 52.9% mIoU for its baseline. This tests whether the effect holds for a different radar representation and a different labeling pipeline.