A segmentation model trained on one kind of scan, such as contrast-enhanced MRI, usually fails on another kind, such as T2-weighted MRI or CT, even when the anatomy is identical. The best current fix has three steps. An image-translation network restyles labeled source scans to look like the target modality. A self-configuring segmenter (nnU-Net, a system that sets its own preprocessing, network shape and training schedule from statistics of the dataset) is trained on these restyled scans. That model then labels real target scans, and the segmenter is retrained on its own labels, a loop called self-training. On the CrossMoDA vestibular schwannoma benchmark this recipe comes within a few Dice points of full supervision. The remaining gap is uneven, though. It is about 4–5 points for the tumor, whose edges are diffuse, and about 2 points for the cochlea, which has crisp edges (CrossMoDA 2021 challenge).
We propose that most of this residual error sits in a thin band around low-contrast boundaries. Self-training makes the band worse, because the model's own boundary mistakes come back as confident training labels. The fix we test is simple to state. The model's own labels on real target scans are split into two parts. The confident interior is supervised as usual. A boundary band is supervised by a softer signal: two views of the same scan must agree. The width of the band is not a hand-tuned hyperparameter. It is measured automatically from the data, in the same spirit as nnU-Net's other self-configured settings. If this works, it closes the tumor–cochlea asymmetry. It should also reduce the "clipping" of tumor fringes that damages any downstream analysis that crops or masks to the predicted region.
Core Claim
We make two claims, and the second depends on the first. The first is diagnostic. After translation plus self-training, target-domain errors are concentrated within a few voxels of the predicted boundary, and more so where local tissue contrast is low. The second is the method. Excluding that band from hard self-labeling, and supervising it instead with a cross-view soft target, improves boundary accuracy on low-contrast structures. The band width must be calibrated per dataset rather than fixed, and the method must leave high-contrast structures roughly unchanged. The cochlea gives us that built-in control.
Where This Sits
Three lines of prior work motivate the proposal.
The CrossMoDA benchmark (Dorent et al.) established translation, self-training and nnU-Net as the winning recipe. It also recorded that the diffuse tumor had larger variance and more outlier failures than the cochlea. That observation is our starting evidence. None of the entries treated boundary voxels differently from interior voxels.
Self-semantic contour adaptation showed that predicted contours transfer across MRI sequences better than raw intensities. It lifted T2→T1 whole-tumor Dice from 4.2% to 59.3%. We borrow its premise that boundary geometry is the more transferable signal. Its method differs from ours in three ways: it uses 2D slices, a fixed architecture, and a separate contour network aligned adversarially. We instead act on the self-training labels of a 3D self-configured model.
RecycleNet feeds decoder features back into the encoder inside nnU-Net. Each pass refines the prediction, and this raised BTCV Dice from 82.99 to 87.80. It was tested only within a single modality. We reuse its repeated passes for a different job: finding which voxels are unstable under target-domain shift.
Two papers constrain the design.
AGENet builds boundary-aware distance fields for few-shot segmentation. Its authors report that the geometric hyperparameters, such as the number of propagation iterations and the speed scaling, destabilize the results when they are mistuned. This is why our band width must be measured, not hand-set.
PnP-AdaNet argues that skip connections leak modality-specific low-level features, and it removes them. We keep nnU-Net's skip connections, because boundaries need high-resolution features. That choice has to be justified empirically. The comparison with a feature-alignment baseline below is partly a test of it.
Region-of-interest work motivates one of our metrics. Mask-weighted pooling (COVID-19 triage, lesion-centric phenotypes) and hard cropping (rheumatoid arthritis scoring) both make downstream models depend on the upstream mask. An under-segmented fringe is therefore lost permanently. We report how much true lesion volume falls outside the predicted region, and treat it as a first-class outcome.
Research Program
Step 1: error-localization study (go/no-go). We first rebuild the CrossMoDA-style pipeline: CycleGAN translation, then nnU-Net, then two rounds of self-training. We then decompose its target-domain errors by signed distance to the true boundary and by local contrast-to-noise, measured in a small neighborhood of each boundary voxel. If less than roughly half of the false voxels lie within a few voxels of the boundary, the premise is wrong. In that case we stop and report the negative result, because a band method cannot help.
Step 2: automatic band calibration. The translated source scans carry real labels and look like the target modality. We can therefore measure boundary error in the target style directly. The band width $w$ is defined as the smallest distance from the predicted boundary beyond which the per-voxel error rate on these labeled translated scans falls to the interior error rate. It is computed once per structure, in millimeters, using the target voxel spacing from the fingerprint. A second definition, used as an ablation, marks band voxels by instability: a voxel is in the band if its predicted class changes across RecycleNet-style repeated passes on the real target scan.
Step 3: training objective. For a real target scan with self-label $\hat{y}$, let $C$ be the core voxels (outside the band $B$) and $B$ the band voxels. Let $p_v$ be the model's predicted class distribution at voxel $v$. Let $q_v$ be a soft target at voxel $v$: the average of the model's predictions on the real scan and on its translation back into the source style, taken from the source-trained model. The loss is
$$\mathcal{L} = \mathcal{L}_{\text{Dice+CE}}(p_C, \hat{y}_C) + \lambda \sum_{v \in B} \mathrm{KL}(q_v \,\|\, p_v),$$
where $\lambda$ weights the band term and $\mathrm{KL}$ is the Kullback–Leibler divergence, a standard measure of mismatch between two probability distributions. We fix $\lambda = 1$ and report sensitivity to it. On the labeled translated scans, boundary voxels keep their exact labels. So the model still learns sharp edges where the labels are trustworthy, and it defers only where the labels are its own guesses.
Baselines
The naive reference is nnU-Net trained on source scans and applied directly to target scans. No reported nnU-Net number exists in the file for this setting. The closest reported direct-transfer figures are 13.2% mean Dice on MM-WHS MRI→CT (PnP-AdaNet), and 4.2% and 6.3% whole-tumor Dice on BraTS 2018 T2→T1 and T2→T1ce (Liu et al.). We rerun this reference ourselves.
The strongest competitor is the CrossMoDA-winning recipe: CycleGAN translation, nnU-Net, and iterative self-training. It is the direct target of our claim, and it held the best reported result, a top median Dice of 87.0% for the tumor and 85.7% for the cochlea against supervised bounds of 92.5% and 87.7% (CrossMoDA 2021). We run it and our method from the same translated images, with the same nnU-Net plans, the same number of self-training rounds and the same GPU budget. The only difference is the loss on real target scans. Beating it means closing at least a third of its measured tumor gap to supervision on our evaluation, while changing cochlea Dice by less than one point.
On BraTS 2018 we compare against contour adaptation (Liu et al.), which reports 59.3% (T2→T1), 63.5% (T2→T1ce) and 82.9% (T2→FLAIR) whole-tumor Dice. We compare both against these reported numbers and against our 3D reimplementation of its contour-hint input inside nnU-Net. The second comparison separates "3D self-configured" gains from "band" gains.
On MM-WHS we compare against PnP-AdaNet's reported 63.9% (MRI→CT) and 54.3% (CT→MRI) mean Dice, and against its CycleGAN baseline at 57.6%.
The key ablation is to ignore the band entirely, with its loss masked out, using a fixed width of two voxels. A second ablation keeps the band but fixes its width instead of calibrating it. A third swaps the distance-defined band for the instability-defined band.
Supervised target-domain nnU-Net serves as the upper bound.
Why not just…?
Why not just use nnU-Net with heavier intensity augmentation? Augmentation can randomize intensities, but it cannot learn intensity inversions between modalities. Every direct-transfer figure in the file collapses to between 4% and 13% Dice without explicit adaptation. The CrossMoDA winners still needed translation on top of nnU-Net's standard augmentations. We include an augmentation-only run for completeness. We expect it to trail the translation baseline by a wide margin.
Why not just threshold self-labels by per-voxel confidence? This is the most serious threat. Confidence filtering already removes many uncertain voxels, and boundary voxels are often among them. Our claim is that spatial structure adds something beyond confidence: errors near low-contrast edges are often confidently wrong after self-training. We test this directly. We run confidence filtering with its threshold chosen so that it discards the same number of voxels as our band. If it matches our method, the band adds nothing and we will say so.
Why not just mask out boundary voxels with a fixed margin? This may capture most of the gain. It is therefore the first ablation, not a straw man. Calibration earns its place only if a single fixed margin is clearly suboptimal on at least one of CrossMoDA, MM-WHS and BraTS. Given AGENet's reported sensitivity to geometric hyperparameters, we expect it to be. If a fixed margin wins everywhere, the contribution shrinks to that simpler finding, which is still worth reporting.
Why not just prompt a foundation model such as SAM? SAM needed dedicated prompt learning to handle ambiguous thin structures (CaPro) and meta-learning to move from CT to MRI (SSM-SAM). Medical vision-language models also misjudge spatial relations under shift (VLM failures). We therefore treat it as out of scope for the main comparison, not as a drop-in alternative.
Proposed Experiments
The primary benchmark is CrossMoDA (ceT1→hrT2), which has two structures of contrasting boundary character. The target validation and test labels are not public. Numbers directly comparable to the reported 87.0% therefore require the official evaluation. If that is unavailable, CrossMoDA contributes only the error-localization study, using labeled translated scans. The quantitative claims then rest on MM-WHS (both directions, with the thin myocardium as the low-contrast structure) and BraTS 2018 (T2→T1, T1ce and FLAIR, with infiltrating edema margins). Both have target labels.
We report Dice, 95th-percentile Hausdorff distance and boundary-band Dice. We also report the clipping rate: the fraction of true lesion volume outside the prediction dilated by one voxel. Every metric is stratified by local contrast-to-noise.
Three checks are decisive. First, gains must grow as local contrast falls. Second, the cochlea and the high-contrast cardiac blood pools must stay flat. Third, the method must beat confidence filtering matched on the number of discarded voxels. If gains are uniform across contrast levels, or confidence filtering matches us, the boundary-ambiguity explanation is wrong, whatever the aggregate Dice.
References
- CrossMoDA 2021 challenge: Benchmark of cross-modality domain adaptation techniques for vestibular schwannoma and cochlea segmentation
- Self-Semantic Contour Adaptation for Cross Modality Brain Tumor Segmentation
- RecycleNet: Latent Feature Recycling Leads to Iterative Decision Refinement
- AGENet: Adaptive Edge-aware Geodesic Distance Learning for Few-Shot Medical Image Segmentation
- PnP-AdaNet: Plug-and-Play Adversarial Domain Adaptation Network with a Benchmark at Cross-modality Cardiac Segmentation
- Computed tomography–based COVID–19 triage through a deep neural network using mask–weighted global average pooling
- Lesion-Centric Latent Phenotypes from Segmentation Encoders for Breast Ultrasound Interpretability
- Interpretable Rheumatoid Arthritis Scoring via Anatomy-Aware Multiple Instance Learning
- CaPro: Curvilinear-aware Prompt Learning with Single Unlabeled Image for Cost-effective Curvilinear Structure Segmentation
- Self-Sampling Meta SAM: Enhancing Few-Shot Medical Image Segmentation With Meta-Learning
- Your other Left! Vision-Language Models Fail to Identify Relative Positions in Medical Images