When a segmentation network trained on one kind of scan (say, contrast-enhanced T1 MRI) is moved to another (high-resolution T2 MRI, or retinal angiography instead of fundus photos), it makes confident mistakes. The best current pipelines handle this with self-training: they guess labels on the unlabeled new-modality scans and retrain on those guesses. When the guesses are wrong, the network learns its own errors. We propose a different way to make the guesses. Run the source-trained model on many contrast-remapped versions of each target scan. Then, at every pixel, take the median of the predicted probabilities rather than their mean, and down-weight pixels where the candidates disagree. The perturbations are random but plausible intensity remappings of the kind that separate MRI sequences from one another. A single bad remapping that produces a hallucinated lesion or a broken cochlea cannot move the median. Disagreement measured by the median absolute deviation tells the student network which pixels not to trust.
The idea brings over a mechanism that works in a different field. Universal Robustness via Median Randomized Smoothing for Real-World Super-Resolution (CertSR) showed that taking a pixel-wise median over about 21 noise-perturbed passes of a super-resolution generator removes erratic hallucinations that mean-smoothing keeps. That result used additive Gaussian and adversarial noise on natural images. Cross-modality shift is not additive noise. It is a change in how tissue maps to intensity. Our central bet is that the perturbation distribution must be moved from pixel noise to contrast space for median smoothing to help here. A second point follows from the design: the median is computed offline, during pseudo-label generation. CertSR's 21× inference slowdown therefore never reaches deployment.
Core Claim
Consider a target image $x$, a set of $K$ random contrast remappings $T_1,\dots,T_K$, and a segmenter $f$ that outputs a per-pixel foreground probability. We build the soft pseudo-label and its per-pixel training weight as
$$\tilde y(p) = \operatorname{med}_{k}\, f(T_k(x))(p), \qquad w(p) = \exp\!\big(-\operatorname{MAD}_k\, f(T_k(x))(p)\,/\,\tau\big),$$
where $p$ indexes pixels, $\operatorname{MAD}$ is the median absolute deviation of the $K$ candidate probabilities around $\tilde y(p)$, and $\tau$ is a temperature. The student is then trained with a $w$-weighted loss toward $\tilde y$.
We claim that this combination, contrast-space perturbation plus robust location and robust spread, produces pseudo-labels with fewer confident false positives and better-preserved thin structures than the same candidates aggregated by mean and variance. We also claim it beats the Gaussian-noise perturbations CertSR used.
We want to be precise about what is and is not new. If $\tilde y$ is thresholded at 0.5, the median gives exactly the majority vote over candidates, which is already a standard ensembling rule. The median differs from the vote only in the soft target and the reliability weight. It differs from the mean whenever one or two candidates are confidently wrong: for candidates {0.45, 0.45, 1.0}, the mean accepts the pixel and the median rejects it. The proposal therefore stands or falls on two things. The first is whether contrast-space perturbation is the right candidate generator. The second is whether robust soft targets and MAD weighting add value beyond a hard majority vote. The experiments are built to separate these two questions.
Relation to Prior Work
Three works motivate the proposal by documenting where errors arise in cross-modality adaptation. The CrossMoDA benchmarks (CrossMoDA 2021; crossMoDA 2021–2023) found that winning pipelines combine image translation, nnU-Net segmentation and iterative pseudo-labeling. The later edition also reports a performance plateau near 88% Dice. MS-MT adds a multi-scale Mean Teacher, meaning an exponential-moving-average copy of the student that supplies consistency targets, on top of segmentation-guided contrastive translation. Every target it produces is a linear average, so a confident student error enters the teacher in proportion to its size. AdaptDiff trains a mask-conditioned diffusion model on noisy pseudo-labels for fundus-to-OCT-A transfer. It shows that pseudo-label noise is the bottleneck, but it uses single-pass pseudo-labels with no aggregation. We keep their pipelines intact and replace only how pseudo-labels are formed.
Two works supply the method. CertSR supplies pixel-wise median aggregation across stochastic passes, together with the finding that dense prediction stabilizes with few samples. We change the perturbation family and use the output as a training target rather than a test-time output. Generalizable Cross-modality Medical Image Segmentation via Style Augmentation and Dual Normalization supplies the perturbation family: Bézier-curve grayscale remappings that simulate sequence changes. That work uses them as training augmentation with mean/variance normalization. We use them at pseudo-label time as a smoothing distribution.
One work constrains the design. Regroup Median Loss shows that median-of-means aggregation of losses resists label noise in classification. It also argues that small-loss and high-confidence selection fails when wrong samples look confident. That argument is why we do not rely on confidence thresholds. The work also suggests that median-based weighting of the loss is a viable alternative to median targets, and we include it as a comparison. The weighted alpha-trimmed mean of Nonreference Medical Image Edge Map Measure offers a continuum between median and mean. We test it because pure medians can erode fine structure.
Research Program
The first component is a pseudo-label generator. A source-trained nnU-Net-style segmenter receives $K$ contrast-remapped copies of each target volume. The remappings are random monotone and non-monotone Bézier curves, optionally mixed with outputs from the translation network's inverse direction. Aggregation is pixel-wise and voxel-wise, across candidates only. It is never applied across a spatial window, so the 0.6 mm-scale cochlea and one-pixel capillaries are not averaged with their neighbors. CertSR's sample-efficiency finding suggests starting at $K=21$ and sweeping $K\in\{5,11,21,41\}$.
The second component drops into existing pipelines. We replace MS-MT's Mean Teacher targets with median-smoothed targets recomputed each self-training round, and we replace AdaptDiff's step-one single-pass pseudo-labels with median-smoothed ones before diffusion training.
The third component is an analysis of why it works. We measure how often the candidate disagreement is dominated by a minority of remappings, which is the regime where the median helps. We compare this with cases where all candidates err together, a systematic contrast error that no aggregation can fix. If the second kind dominates, the method cannot help much, and we will report that.
Baselines
The naive reference point is the source-trained segmenter applied to translated or raw target scans with no adaptation. AdaptDiff reports 73.16% Dice for this on OCTA-500(6M), against an 81.94% AdaptDiff result and an 84.78% supervised oracle. We reproduce these numbers before adding anything.
The strongest existing method on the MRI task is the translation-plus-self-training family that won CrossMoDA. The 2021 winner reports a median Dice of 88.4% for vestibular schwannoma and 85.7% for cochlea, with a supervised reference of 92.5% and 87.7% (CrossMoDA 2021). MS-MT reports 83.8% and 81.4% mean Dice, with 0.55 mm and 0.26 mm ASSD, on the 2022 validation set. Because challenge entries differ in engineering effort, we will not compare against leaderboard numbers directly. Instead we reimplement the MS-MT pipeline with the same translator, segmenter, training schedule and data split. The only change is the pseudo-label rule. Beating it means a gain in cochlea Dice and ASSD that holds across three seeds, because cochlea is the thin structure where hallucinations hurt most. VS Dice must not drop.
On OCTA-500 the strongest baseline is AdaptDiff run with our compute budget. Beating it means exceeding 81.94% Dice with a reduced vessel-connectivity error, measured as the number of fragments per image.
The key ablations hold the $K$ candidates fixed and change only the aggregation rule. The rules are mean with variance weighting, hard majority vote, alpha-trimmed mean, and median with MAD weighting. A second ablation holds the aggregation rule fixed and changes the perturbation family: Gaussian noise as in CertSR, versus contrast remapping, versus both. All variants get the same tuning budget for $\tau$, $\alpha$ and the Gaussian noise level $\sigma$, selected on a held-out labeled target subset of equal size for every method.
Why not just…?
Why not just keep the Mean Teacher, or average the candidates? This is the main competitor. We do not claim the answer in advance. CertSR found mean-smoothing keeps hallucinations in generative outputs, but segmentation probabilities are bounded in [0,1]. That bound limits how far a single outlier can pull the mean. The aggregation ablation settles this directly. If median-plus-MAD does not beat mean-plus-variance on the same candidates, the aggregation claim fails, and whatever gain remains belongs to contrast smoothing alone.
Why not just threshold confidence? Regroup Median Loss argues that confidence-based selection breaks when errors are confident. Cross-modality contrast errors are often of this type. We still run a strong version, keeping pixels whose single-pass probability falls outside $[0.05, 0.95]$, and report its precision on false-positive pixels against MAD weighting.
Why not just take a majority vote across test-time augmentations? For hard labels the median is exactly a majority vote. The real question is therefore whether soft median targets and MAD weights add anything, and whether contrast remappings beat standard flips and rotations. Both appear in the ablations. If majority vote over contrast remappings matches our full method, we will report the simpler rule as the finding.
Why not just train with Bézier augmentation instead? Dual Normalization uses these remappings as augmentation and reaches only 54.44% Dice on BraTS T2-source generalization, well below adapted pipelines. That supports using target scans. We still include an augmentation-only arm with identical remappings, to show that the remappings must be used at pseudo-label time and not only at training time.
Why not just apply a spatial median filter to the outputs? A spatial window blends neighboring voxels and erases structures one or two voxels wide. We include a 3×3×3 post-hoc filter and expect it to lower cochlea and capillary Dice. If it does not, then thin-structure preservation is not a reason to prefer our method.
Decisive Checks
The experiments use CrossMoDA as the primary setting: 105 labeled ceT1 scans, 105 unlabeled hrT2 scans, and the multi-site 2022/2023 cohorts, scored by Dice and ASSD. OCTA-500(6M) fundus-to-OCT-A transfer, following the AdaptDiff protocol, is the secondary setting, scored by Dice and fragment count.
Four results would decide the proposal. First, on the reimplemented MS-MT pipeline, median-plus-MAD targets must beat mean-plus-variance targets built from the same contrast-perturbed candidates on cochlea Dice and ASSD. Second, contrast-space perturbations must beat Gaussian perturbations under the same aggregation. Third, the gain must concentrate on voxels where a minority of candidates disagreed. We check this by stratifying error reduction by the fraction of dissenting candidates. A gain spread uniformly would suggest generic ensembling rather than outlier rejection. Fourth, the gain must persist on the multi-site 2023 cohorts, where scanner artifacts make intensity outliers most common.
A null result on the first check, with a positive result on the second, would still be useful. It would turn the paper into a study of which perturbation family to use for pseudo-labels, and it would show that the robust-statistics angle is unnecessary.
References
- Universal Robustness via Median Randomized Smoothing for Real-World Super-Resolution
- CrossMoDA 2021 challenge: Benchmark of Cross-Modality Domain Adaptation techniques for Vestibular Schwnannoma and Cochlea Segmentation
- crossMoDA Challenge: Evolution of Cross-Modality Domain Adaptation Techniques for Vestibular Schwannoma and Cochlea Segmentation from 2021 to 2023
- MS-MT: Multi-Scale Mean Teacher with Contrastive Unpaired Translation for Cross-Modality Vestibular Schwannoma and Cochlea Segmentation
- AdaptDiff: Cross-Modality Domain Adaptation via Weak Conditional Semantic Diffusion for Retinal Vessel Segmentation
- Generalizable Cross-modality Medical Image Segmentation via Style Augmentation and Dual Normalization
- Regroup Median Loss for Combating Label Noise
- Nonreference Medical Image Edge Map Measure