Medical segmentation models trained on one kind of scan, such as CT, are routinely deployed on another, such as MRI, without any labels from the new scanner. The strongest methods for this setting randomize image intensities during training so the network learns anatomy instead of scanner appearance. They are judged almost entirely by Dice, an area-overlap score. Dice is dominated by an object's interior, so it barely registers errors along its outline. Outlines are exactly where modality shift should hurt most. A tissue border that shows a sharp intensity step in CT can fade to near-zero contrast in MRI, or the reverse. Current methods either ignore boundaries entirely or feed classical edge maps to the network, and those edge maps collapse precisely where contrast disappears.
This note proposes to measure the problem first and then attack it directly. Many anatomical borders will be invisible at test time, so the network should learn during training to place a border even when the image offers no local evidence for it. We do this with a training-time intervention, boundary contrast collapse. It uses the ground-truth outline to erase or invert the local intensity step on random stretches of the true boundary. A boundary prediction head supervised by label-derived distance maps then still tells the network where the border is. The model is forced to infer contours from shape and surrounding context instead of local gradients. At deployment, local gradients are the cue least likely to survive a change of scanner.
Core Claim
We make two claims, and each can be falsified on its own. The diagnostic claim is that in single-source cross-modality generalization, boundary metrics degrade much more than Dice does. That hidden degradation should concentrate on boundary segments whose local contrast in the target modality is low. The method claim is that removing boundary contrast during training in a way tied to the label, with decoy removals elsewhere so the network cannot learn "blurred band means edge," improves boundary accuracy in those low-contrast segments. It should do so beyond what a boundary loss or random local corruption adds to a strong intensity-randomization baseline, and without harming high-contrast segments.
Where This Sits in Prior Work
The motivating evidence comes from three places. First, Boundary IoU showed that area metrics shield boundary errors. It scores overlap only within a thin band around contours, and on COCO it found Mask R-CNN falling from 66.0 mask AP to 25.9 boundary AP on large objects. Second, the CHAOS challenge reported that even high-Dice abdominal models had maximum surface distances of roughly 20–30 mm. Third, the MM-WHS challenge found thin structures such as the great vessels to be major cross-modality failure modes. None of these measured boundary quality in the zero-target-label generalization setting, and we will.
The method we build on is the causal augmentation of Ouyang et al.. It passes images through random shallow convolutional networks (called GIN) to produce varied nonlinear contrast. It then blends differently transformed copies with smooth random fields (called IPA) to decouple foreground from background texture, reaching 86.31% Dice on abdominal CT-to-MRI against a 77.31% baseline. Its interventions are global and low-frequency and never target the boundary. GIN changes intensities globally, which means it tends to preserve a detectable step at most borders. The step's sign or size may change, but it usually remains. Our intervention is local and removes the step.
The constraining evidence comes from EGSDG. It feeds Canny and Sobel edge maps as extra input channels under Bézier intensity augmentation, reaching 64.78% Dice on BraTS'19 cross-sequence. Its authors state that classical edge maps turn noisy or fail in low-contrast targets and on small structures. We take that as a negative result, and it rules out any design in which the boundary signal is computed from the image at test time. CTO has the same dependence. It injects fixed Sobel gradients into the decoder and helps blurry pancreas and stomach borders in-domain, but it assumes gradients mark anatomy, and that assumption is what breaks under modality shift.
Two further lines of work supply the boundary head. E2Net contributes normalized distance-map supervision, which avoids the extreme class imbalance of binary edge targets. RPCNet contributes a duality loss, which ties the spatial gradient of the predicted mask to the predicted edge map. We use RPCNet's loss without its eight-step iterative module, so inference stays feed-forward.
AGENet also concentrates learning at borders across modalities, using geodesic distance fields. However, it needs labeled support images from the target modality, so it answers a different, few-shot question. SIFA aligns MR and CT through image translation and adversarial training. It uses unlabeled target data, which our setting forbids, and it has no boundary constraint.
Research Program
The first component is a boundary audit of existing cross-modality generalization methods. We retrain an unaugmented baseline, standard intensity augmentation, GIN+IPA, and EGSDG under identical protocols. For each, we report Dice, a 3D Boundary IoU with a band width fixed in millimetres, the combined score min(Dice, Boundary IoU), the 95th-percentile Hausdorff distance (HD95), and average symmetric surface distance. The combined score closes the loophole the Boundary IoU paper warns about, where a hollow ring can score well on boundary overlap alone.
The audit also includes a contrast stratification. We split every ground-truth target contour into short segments and compute each segment's local contrast in the target image. Local contrast here means the absolute difference in mean intensity just inside versus just outside the border, normalized by local intensity spread. We then bin segments by that contrast and report surface error per bin. If the diagnostic claim is right, boundary error in the lowest-contrast bins should dwarf the Dice gap for every existing method.
The second component is the method. The backbone is a standard 2D U-Net, trained exactly as in the GIN+IPA protocol, with two added pieces.
The first added piece is a boundary head. It regresses a normalized signed distance map computed from the label. It is coupled to the mask head by a duality loss that penalizes disagreement between the mask's spatial gradient and the predicted edge. Because this target comes from labels rather than image gradients, it is defined even where the image shows no border.
The second added piece is boundary contrast collapse, applied after GIN. Let $x$ be the augmented image and $m$ a smooth mask supported on randomly chosen arc segments of a band of width $w$ around the true contour. Let $G_\sigma * x$ be $x$ blurred with a Gaussian of width $\sigma$ comparable to $w$, and let $\alpha \in [-1, 1]$ be a random strength. The intervention is
$$x' = x + \alpha\, m \odot \left( G_\sigma * x - x \right),$$
where $\odot$ is elementwise multiplication. With $\alpha$ near 1, the intensity step across the selected border segments is flattened. With negative $\alpha$, the step is sharpened, and for large $|\alpha|$ it overshoots, which roughly mimics contrast changes between modalities.
To prevent the network from learning that a blurred band signals a boundary, identical operations are applied at an equal number of decoy locations away from any contour. We then verify the shortcut is absent with a probe experiment: we run the trained model on decoy-only images and check that no boundary is predicted at the decoys.
The third component is a failure analysis aimed at a predictable risk. When the image is uninformative, the network may fall back on memorized typical shapes. This is the same failure Wolf et al. document for multimodal medical models. Such fallback is acceptable for healthy organs but dangerous for pathology. We therefore test on tumors, where shape priors are weak, and on healthy organs separately. We also check whether low-contrast gains come with lower accuracy on atypical anatomy.
Baselines
The naive reference point is a U-Net trained on the source modality with only standard spatial augmentation and tested directly on the target. It shows how much any method recovers. In the Ouyang et al. abdominal CT-to-MRI protocol, this kind of baseline scored 77.31% Dice.
The strongest existing method on this question is GIN+IPA from Ouyang et al.. It reports 86.31% Dice on abdominal CT-to-MRI, 85.01% on cardiac bSSFP-to-LGE, and 70.37% on cross-center prostate. We found no reported Boundary IoU, HD95, or surface-distance result for it on Lacuna, so our audit will produce the first boundary numbers. We will rerun it with its released protocol, the same backbone, and the same number of training iterations as our method, and first confirm we reproduce its Dice to within about one point. Our method is GIN+IPA plus our additions, so every comparison against it isolates the additions. Beating it means a statistically significant gain in lowest-contrast-bin surface error and in min(Dice, Boundary IoU), paired across target volumes, with Dice not falling.
The edge-input competitor is EGSDG. It is the only prior single-source method built around boundaries, reporting 64.78% Dice on BraTS'19. Its prostate and MMWHS numbers are not given in the file. We will run it with its Canny/Sobel channels on the same splits, and also with GIN in place of its Bézier augmentation. The GIN variant separates the value of edge channels from the value of the augmentation choice.
Two ablations isolate the key ingredient, and both keep everything else fixed. The first is GIN+IPA plus the boundary head and duality loss, with no contrast collapse. This asks whether a label-derived boundary target alone is enough. The second applies contrast collapse at random locations not tied to the contour, which asks whether the gain comes from generic local corruption. All variants get the same hyperparameter search budget, the same number of search trials over learning rate and loss weights, selected only on held-out source-domain validation data, because target labels are off-limits. Results are reported over three seeds.
Why not just…?
Why not just add a boundary loss to GIN+IPA? This is the most serious threat, and it is included as the first ablation. A distance-map or duality loss makes the network care about outlines. But during training nearly every border in the augmented images still shows some intensity step, so the network can satisfy that loss by reading local gradients. That cue is exactly the one that vanishes at test time on low-contrast segments. If the boundary-loss-only variant matches the full method in the lowest-contrast bin, contrast collapse is unnecessary and we will say so. The prediction is that the two variants look similar on high-contrast segments and separate on low-contrast ones.
Why not just feed classical edge maps as extra channels, as EGSDG does? The EGSDG authors themselves report that Canny and Sobel maps become noisy or fail to fire on low-contrast targets and small structures. CTO makes gradient operators work only in-domain, where the scanner is fixed. The contrast-stratified audit tests this directly: an edge-channel method should gain on high-contrast segments and gain nothing where contrast is near zero. That is the regime our method targets.
Why not just use random local corruption such as cutout, local blur, or random-field blending? If erasing local evidence anywhere teaches the same robustness, then tying the intervention to the label is decoration. The second ablation, collapse at random off-contour locations, settles this. IPA itself is a global random-field blend, and it already sits inside every compared variant. Only the full method removes contrast specifically at true borders.
Why not just prompt a foundation model like SAM? HQ-SAM documents that base SAM's outlines are coarse, at 52.8 boundary IoU on DIS, and that fixing this required new trained tokens and curated fine masks. SSM-SAM reports that SAM does not reliably segment across medical modalities without further adaptation. SAM also needs prompts, which is a different deployment setting. We will still report box-prompted SAM on the target test sets as an external reference, not as a like-for-like competitor.
Proposed Experiments and Decisive Checks
The benchmarks follow the Ouyang et al. protocols. Abdominal organ segmentation runs CT source to T2-SPIR MRI target, the MRI being CHAOS data. Cardiac MRI runs bSSFP to LGE. MMWHS cardiac segmentation runs CT to MRI and MRI to CT, where thin myocardium and great vessels are known failure points. Cross-center prostate MRI trains on one center and tests on five. BraTS'19 cross-sequence adds tumors with weak shape priors, following the EGSDG setup.
The SIFA numbers on MMWHS, 73.0% Dice MR-to-CT, are cited only for context, because SIFA uses unlabeled target images and we do not.
Three checks decide the project. First, the audit must show that for GIN+IPA the relative gap between source and target is substantially larger for Boundary IoU and HD95 than for Dice, and concentrated in low-contrast bins. If boundary and area metrics degrade in proportion, the premise fails. Second, the full method must beat both ablations in the lowest-contrast bin on at least three of the five benchmarks, while staying within noise of GIN+IPA on the highest-contrast bin. A loss on high-contrast segments would mean the model learned to distrust real edges. Third, the decoy probe must show no boundary predictions at blurred off-contour locations, and BraTS tumor boundaries must not worsen relative to GIN+IPA. Worse tumor boundaries would mean the gains on organs came from shape memorization rather than contextual inference.
References
- Boundary IoU: Improving Object-Centric Image Segmentation Evaluation
- CHAOS Challenge - combined (CT-MR) healthy abdominal organ segmentation
- Evaluation of Algorithms for Multi-Modality Whole Heart Segmentation: An Open-Access Grand Challenge
- Causality-Inspired Single-Source Domain Generalization for Medical Image Segmentation
- Train Once, Deploy Anywhere: Edge-Guided Single-source Domain Generalization for Medical Image Segmentation
- Rethinking Boundary Detection in Deep Learning Models for Medical Image Segmentation
- E2Net: An Edge Enhanced Network for Accurate Liver and Tumor Segmentation on CT Scans
- Joint Semantic Segmentation and Boundary Detection Using Iterative Pyramid Contexts
- AGENet: Adaptive Edge-aware Geodesic Distance Learning for Few-Shot Medical Image Segmentation
- Synergistic Image and Feature Adaptation: Towards Cross-Modality Domain Adaptation for Medical Image Segmentation
- Your other Left! Vision-Language Models Fail to Identify Relative Positions in Medical Images
- Segment Anything in High Quality
- Self-Sampling Meta SAM: Enhancing Few-Shot Medical Image Segmentation With Meta-Learning