Distilling Only What the Student Can See: Visibility-Gated Boundary Distillation from Contrast-Enhanced to Non-Contrast Scans

Many medical segmentation models must run on cheap, safe scans, such as MRI without a contrast agent or CT without iodine. During training, though, hospitals often also have the contrast-enhanced version of the same patient, where tumor margins and vessel walls stand out sharply. The natural move is to train a "teacher" network on the enhanced scan and have a "student" network, which sees only the non-contrast scan, imitate it. The danger is that some boundaries the teacher sees are physically absent from the student's input. Two tissues can attenuate X-rays or relax magnetically almost identically, so no amount of model capacity recovers the edge. A student forced to copy the teacher's sharp edge there learns to invent confident boundaries. This is the worst failure mode for radiotherapy planning or surgical margins.

We propose to distill, voxel by voxel, only the part of the teacher's output that can be predicted from the student's input. A small probe network, trained with cross-fitting (fit on one data fold, evaluated on another), tries to predict the teacher's boundary logits from the non-contrast scan alone. Where the probe succeeds, the student imitates the teacher's sharp edge. Where it fails, the student imitates the probe's prediction, which is the best estimate of the teacher's output given only the non-contrast scan. That estimate is soft precisely where the anatomy is invisible. The resulting student should be sharp where the edge is recoverable and honestly uncertain where it is not. Existing boundary distillation methods give the second region no special treatment.

Core Claim

Generalized distillation, the framework of Unifying distillation and privileged information in which a teacher trained on training-only "privileged" inputs supervises a student that sees only the cheap input, helps when the teacher's output is easier to learn from the cheap input than the raw labels are. Our claim is that in low-contrast segmentation this condition holds in some boundary regions and fails in others, and that it can be estimated per voxel. Let $t(v)$ be the teacher's boundary logit at voxel $v$. Let $p(v)$ be the cross-fitted probe's prediction of $t(v)$ from the non-contrast input. Let $e(v)$ be the probe's held-out error, smoothed over a small neighborhood. The student is trained toward the mixed target

$$q(v) = g(v)\,\sigma(t(v)) + (1-g(v))\,\sigma(p(v)), \qquad g(v) = \exp(-e(v)/\tau),$$

where $\sigma$ is the sigmoid and $\tau$ is a temperature set on validation data. The loss is a KL divergence between $q$ and the student's prediction, computed only inside a narrow band around the ground-truth contour. Following the edge/body split of BPKD, the band loss is normalized by the band area, and a separate body term handles the interior. The new ingredient is the gate $g$. It decides how much of the teacher's sharpness the student is allowed to inherit.

Relation to Prior Work

Two papers come closest to this setting.

BPKD splits distillation into an edge stream and a body stream, using ground-truth edge masks. It shows that whole-image distillation lets confident interior signal leak across and blur boundaries. That finding motivates our band-restricted loss, and we reuse its mask normalization. BPKD, however, is same-modality model compression on sharp-edged RGB scenes, where the teacher's edge is always learnable by the student in principle. It has no notion of an unrecoverable edge.

Boundary and Relation Distillation transfers boundaries derived from inter-pixel affinities, again RGB to RGB. It shares the same blind spot.

On the privileged side, three methods transfer knowledge across modalities without explicit boundary handling:

  • The gadolinium-free nasopharyngeal carcinoma model distills contrast-enhanced T1 features into a T1/T2 student. It reaches internal AUC 0.95 against 0.97 for the full-contrast model. It is a classifier, though, and the authors' worry about infiltrative margins is never measured with a surface metric.
  • Curriculum disentanglement for semi-paired brain MRI separates anatomy from contrast style across MRI sequences. It has no boundary-specific term.
  • Hard Pixels Mining uses depth prediction error to reweight hard pixels in an RGB model. Its weighting rule is the opposite of ours: it pushes harder on pixels where the auxiliary signal is hard to predict. Our hypothesis is that in medical low-contrast settings, pushing harder at unpredictable voxels is exactly what produces hallucinated edges, and the experiments test this directly.

Privileged Prior Information Distillation for Image Matting focuses distillation on a transition zone chosen by teacher attention. It is the nearest relative in spirit. Its zone, however, is chosen by where the teacher is uncertain, whereas ours is chosen by whether the student could ever agree with the teacher.

The boundary-ambiguity literature constrains the design. Combating Uncertainty with Novel Losses shows that sharpening losses lift left-atrium Dice from 82.01% to 92.24%, but on gadolinium-enhanced MR where the wall is visible. NA-UNETR for the LAD artery reaches only 45.64% Dice on non-contrast CT after contrast pretraining and states that performance is "fundamentally limited by the visibility of the anatomy." We take that limit seriously rather than trying to engineer past it.

Research Program

The primary testbed is BraTS2020, which is paired by construction and therefore allows voxel-aligned distillation. The teacher sees all four sequences, including contrast-enhanced T1ce. The student sees T1, T2 and FLAIR. The three tumor subregions make a built-in visibility experiment:

  • Whole tumor. Defined largely by FLAIR hyperintensity, so it is mostly visible to the student.
  • Enhancing tumor. Defined by contrast uptake, so it should be largely invisible without T1ce.
  • Tumor core. Expected to fall in between.

If the gate works, its average value should be high on whole-tumor boundaries and low on enhancing-tumor boundaries, and this ordering is checkable without any new annotation.

All networks share an nnU-Net-style 3D backbone, so that differences come from the training signal and not the architecture. The probe is a deliberately small head on a frozen label-only student encoder, trained with five-fold cross-fitting. This ensures $e(v)$ measures generalizable predictability rather than memorization. We will also study how the gate behaves as probe capacity grows. If a larger probe makes everything look predictable, the gate is measuring memorization and needs stronger cross-fitting. If the gate is stable across probe sizes, it is measuring something physical.

A second, harder regime uses the pancreas. Many PanTS-style multicenter cohorts (PanTS) include multiphase CT. Where a paired non-contrast phase exists for a subset of patients, we register it to the venous phase and repeat the study, accepting registration error as an extra source of unpredictability that the gate should also absorb. If such paired phases are not available in the released data, this regime is dropped rather than simulated.

We do not run voxel distillation on LAD-SEG. Its contrast counterpart, ImageCAS, comes from different patients, so the method does not apply. LAD-SEG serves only as an external reference for how far sequential transfer gets.

Baselines

The naive reference is a label-only student trained on T1/T2/FLAIR with Dice plus cross-entropy, using the same backbone, schedule and augmentation as every other arm. A second reference is the full-modality teacher, which sets the ceiling.

The strongest existing competitor is BPKD applied cross-modally. It uses the same edge band, pre- and post-mask normalization, and channel-wise body distillation, but with the gate fixed at one, so the student copies the teacher everywhere in the band. This is exactly the obvious combination of the two research directions. Beating it is the central requirement. BPKD reports up to 24.33% mIoU gains and +7.58% on Cityscapes trimap boundary metrics in its own same-modality setting, but there is no reported result found on Lacuna for cross-modal BPKD on BraTS.

We also run three other comparisons:

  • Whole-image logit distillation. This shows how much the band restriction alone contributes.
  • The label-only student with Overlap Loss and Focal Positive Loss from Combating Uncertainty with Novel Losses. This is the strongest no-privileged-information boundary method in the file.
  • Hard-pixel weighting in the style of Hard Pixels Mining. Here the gate is inverted, so that weight increases with $e(v)$.

As a published anchor for BraTS2020 whole tumor, curriculum disentanglement reports average Dice of 82.91% against an 81.31% baseline. That was a semi-paired, missing-modality protocol rather than ours, so we cite it as context and rerun it under our split rather than compare raw numbers.

Two ablations isolate the key ingredient. The first replaces the predictability gate with a gate built from teacher entropy, which is matting-style focusing. The second replaces it with a random gate matched in mean value.

Every arm gets the same data split, the same GPU-hour budget, and the same number of hyperparameter trials on a held-out validation fold.

Beating cross-modal BPKD means three things together:

  • equal or better whole-tumor HD95 and normalized surface Dice;
  • significantly lower boundary-band calibration error on enhancing-tumor and tumor-core boundaries;
  • a lower "false confident edge" rate, defined as the fraction of band voxels where the student is above 0.9 confidence and wrong.

A method that only wins on Dice does not count.

Why not just…?

Why not just apply BPKD across modalities? This is the real threat. On whole tumor it may match us, because nearly every boundary voxel there is recoverable and the gate stays near one. The disagreement is predicted on enhancing tumor. There, BPKD should produce sharp but misplaced edges, which shows up as good-looking HD95 on some cases, poor calibration, and a high false-confident-edge rate. If cross-modal BPKD matches us on enhancing-tumor calibration as well, the gate adds nothing and the claim fails.

Why not just pretrain on the contrast-enhanced data and fine-tune on non-contrast? Sequential transfer is what NA-UNETR does, reaching 45.64% Dice and 38.16 mm HD95 on LAD-SEG. It gives a starting point but no per-case boundary supervision. We include a pretrain-on-four-sequences, fine-tune-on-three arm with the same budget. We expect it to land near the label-only student on boundary metrics, because the contrast-dependent features it learned have nothing in the student input to bind to.

Why not just synthesize a virtual contrast image and segment that? The nasopharyngeal carcinoma study found that synthesis methods lost infiltrative margins, and feature distillation beat them by up to 69.8% in accuracy comparisons. Synthesis also faces the same visibility problem in a hidden form: a generator must invent the enhancing rim. We will measure hallucinated-edge rate on a pix2pix-style synthesis arm to make this concrete rather than assumed.

Why not just gate on where the label-only student is wrong? This is simpler and partly correlated with our gate. However, student error mixes capacity limits, annotator noise and true invisibility. Our gate asks only whether the teacher's signal is predictable from the input. The student-error gate is run as an additional ablation. If it performs as well as the cross-fitted probe, we will adopt it and say so.

Proposed Experiments and Decisive Checks

All arms are evaluated on BraTS2020 using:

  • Dice, HD95 and normalized surface Dice per subregion;
  • boundary-band expected calibration error within 3 voxels of the contour;
  • the false-confident-edge rate;
  • Spearman correlation between the gate value and local intensity contrast across the boundary in the student's input sequences, as a sanity check that the gate tracks physical visibility.

The first decisive check is the subregion ordering: mean gate on whole-tumor boundaries above tumor core above enhancing tumor. The second is that inverting the gate, as hard-pixel mining would, increases the false-confident-edge rate on enhancing tumor relative to our method. The third is the cross-modal BPKD comparison, using the win condition defined in the Baselines section.

A practical deliverable follows if all three hold. The probe's soft output marks a per-voxel band where the non-contrast scan cannot support a sharp contour. A clinician can then decide whether that patient needs a contrast scan after all.