Hospitals hold huge archives of unlabeled scans from many machines, but expert outlines exist for only a few scans, usually from one site. Semi-supervised learning trains a model on those few labeled scans and then lets it teach itself on the unlabeled ones. In this regime the model's own predictions on unlabeled scans, called pseudo-labels, become training targets. We argue that pseudo-labels fail at unfamiliar sites for a specific reason. The model has learned the fine-grained noise and reconstruction texture of the labeled site's scanner, and when that texture changes its predictions change too, even though the anatomy is identical. Errors then compound, because the student model trains on the teacher's texture-driven mistakes.
The proposal has two parts. The first is a medical "texture transplant" probe. We split each scan into an edge-preserving smooth image, which keeps anatomy and organ boundaries, and a residual, which holds the scanner-specific fine texture. We then swap residuals between scans from different vendors. This yields cue-conflict images: the same anatomy rendered with another scanner's texture. Such images are otherwise hard to build in medicine without destroying diagnostic content. The second part uses these transplants as the perturbation for consistency training on unlabeled multi-site data. The teacher sees the original scan, the student sees the same anatomy with another site's real residual, and pseudo-labels are kept only where the prediction survives the swap. The method is useful only if real cross-site residuals beat cheaper synthetic noise and beat existing contrast-style mixing. The experiments are designed to test exactly that, and they can show that the simpler options are enough.
Core Claim
Our hypothesis is that, in mixed-domain semi-supervised segmentation of MRI, much of the pseudo-label error at unseen vendors comes from sensitivity to high-frequency texture that does not carry anatomical edges. Contrast and intensity differences explain only part of it. If this holds, two things follow. A model's measured sensitivity to residual transplants should predict its per-site pseudo-label error. And enforcing invariance to real transplanted residuals should close part of the remaining cross-vendor gap that intensity augmentation and amplitude mixing leave open.
Three observations in the file motivate the hypothesis. First, generalization to the lower signal-to-noise Canon vendor in the M&Ms challenge stays systematically worse even after the intensity augmentations that won it. Second, models in AMOS drop by more than 20% on MRI from unseen scanners. Third, benchmarking of segmentation robustness found that segmentation networks are most fragile to fine, high-frequency corruptions such as sensor noise and comparatively resilient to corruptions that keep shape contours.
Two prior results constrain the design. Few-shot texture-bias work suppresses high frequencies with a Difference-of-Gaussians filter. FRCNet finds that high-frequency edge content is needed to resolve fine lesion boundaries. Read together, these say that blunt low-pass filtering is the wrong tool. The decomposition has to keep edges in the smooth part and remove only edge-free texture.
Research Program
For an image $x$, let $S(x)$ be the output of an edge-preserving smoother; non-local means and a learned denoiser are the candidates. Let $r(x) = x - S(x)$ be the residual. Given an unlabeled scan $x_j$ from a different site, the transplanted view is
$$\tilde{x} = S(x) + \alpha \, \frac{\sigma(x)}{\sigma(x_j)} \, r(x_j),$$
with the terms defined as follows:
- $\alpha$ is sampled in $[0.5, 1.5]$.
- $\sigma(\cdot)$ is the residual's standard deviation in foreground tissue, so the rescaling preserves the source intensity scale.
- The residual is cropped or tiled spatially, because exact anatomical alignment is unnecessary for texture.
Training follows a mean-teacher setup. The teacher predicts on $x$ and the student predicts on $\tilde{x}$. The consistency loss is applied per voxel, weighted by $w_v = \exp(-\lambda \, d_v)$. Here $d_v$ is the teacher's disagreement at voxel $v$ between $x$ and its own prediction on a transplanted copy, and $\lambda$ sets how quickly weight falls off with disagreement. Voxels whose label flips under a texture swap are therefore down-weighted rather than hard-thresholded. This choice follows DyCON's evidence that binary thresholds discard boundary voxels.
The main technical risk is that the residual carries real anatomy, such as thin myocardium or vessels. We handle this before training starts. On the labeled site only, we pick the smoother's strength as the strongest setting at which the residual's gradient magnitude is uncorrelated with ground-truth boundary maps. We report this correlation for every dataset. The probe that comes out of this step, residual-swap Dice stability per site, is a reusable contribution independent of the training method.
This differs from the closest work in two ways. MiDSS swaps low-frequency Fourier amplitude, which captures global contrast and style, and copy-pastes regions to build intermediate domains. It never isolates the edge-free fine texture and never measures the model's reliance on it. DAC uses co-training with patch-level self-supervised tasks and has no texture-specific perturbation. Shape-prior methods, namely VAE shape-regularized adaptation and TASD, constrain outputs toward learned shapes. TASD's own authors concede that this fails on amorphous anatomy. Our perturbation constrains the input side and assumes no shape template.
Baselines
The naive reference is a supervised nnU-Net-style U-Net trained only on the labeled site, tested directly on all vendors. The M&Ms challenge reports a roughly 6% cross-vendor drop for baseline models of this kind. The second reference is the same network inside a plain mean teacher with standard weak augmentation, which isolates what semi-supervision alone contributes.
The strongest practical competitor is a mean teacher whose student view uses the M&Ms recipe: gamma, brightness, i.i.d. noise synthesis and histogram matching. That recipe closed the cross-vendor drop to 1.5% in M&Ms and beat feature-alignment methods there.
The strongest published method on our exact setting is MiDSS. It reports 87.66% average Dice on the mixed-domain Fundus benchmark against an 88.46% fully labeled upper bound, and a 13.57% Dice improvement over prior methods on Prostate. The file gives no MiDSS per-vendor M&Ms numbers, so no reported result was found on Lacuna for that comparison and we will rerun it. We also include DAC, which reports 76.35% optic-cup Dice under single-site-labeled training with 10% labels, and TASD, which reports 81.1% average prostate Dice when trained on NCI-ISBI13 and tested on five unseen sites.
Fairness rules apply to every method. All share the same backbone, labeled and unlabeled splits, iteration count and tuning budget, with 20 random-search trials over loss weights on a held-out labeled-site validation set. No target-vendor labels are used for tuning. Beating MiDSS means a higher worst-vendor Dice averaged over three seeds with non-overlapping 95% bootstrap intervals, without lowering the labeled-site Dice.
Three ablations isolate what drives any gain:
- Spectrum-matched synthetic noise replaces the real transplanted residual with colored noise whose power spectrum matches a randomly drawn target site. It tests whether real residual structure matters beyond its spectrum.
- Amplitude mixing combined with transplants tests whether the two perturbations are complementary.
- Transplant consistency without the stability weighting tests whether the weighting is needed.
Why not just…?
Why not just use the M&Ms intensity augmentations inside a mean teacher? This is the real threat, because it is cheap and empirically strong. Its noise synthesis is i.i.d. Gaussian. Real MRI reconstruction texture is spatially correlated and varies across the image, and the persistent Canon gap suggests that the i.i.d. model misses something. The spectrum-matched ablation settles the question. If colored synthetic noise matches real transplants, the lesson is "use better parametric noise", and we will report that as the result rather than the transplant method.
Why not just use MiDSS-style Fourier amplitude mixing? Amplitude swaps alter the global contrast that dominates the low-frequency spectrum and leave fine texture largely untouched. If the probe shows that MiDSS-trained models are already stable under residual swaps, our hypothesis fails for them. The combined ablation shows whether the two perturbations add up or overlap.
Why not just low-pass filter inputs, as the few-shot texture-bias work does? FRCNet's ablations indicate that high-frequency edges matter for boundaries, and blur removes exactly those edges for thin structures such as the myocardium. We include a Difference-of-Gaussians input-filtering baseline and report boundary distance (HD95, the 95th-percentile Hausdorff distance between predicted and true boundaries) separately from Dice, where we expect blur to hurt.
Why not just add a shape prior? On compact organs this is a strong option, and TASD's 81.1% prostate result shows it. Shape priors, however, need a template distribution and break on irregular anatomy. Our perturbation needs no template. On AMOS MRI we report small irregular organs separately to test this distinction.
Proposed Experiments
The primary setting is M&Ms in its mixed-domain semi-supervised form, with labels from a fraction of one vendor's cases and unlabeled cases from all four vendors. The primary metric is per-vendor and worst-vendor Dice and HD95 for the left ventricle, right ventricle and myocardium, with Canon reported separately. Prostate multi-site MRI is run in both the MiDSS and the TASD protocols. Fundus serves as a planned weak case. Its shift is mostly color and camera optics rather than noise texture, so we predict little gain there, and a large gain would undercut our mechanistic story.
The scaling test is AMOS MRI. We train with labels from some scanner models and treat the rest as unlabeled and held-out, and we report regular organs (liver, spleen) separately from irregular ones (duodenum, adrenal glands).
The first decisive check is diagnostic and runs before any method comparison. Across all baseline models and sites, we compute the rank correlation between residual-swap prediction stability and pseudo-label Dice on unlabeled sites, measured against held-out labels. A correlation near zero refutes the premise and ends the method track. The second check is the spectrum-matched noise ablation described above. The third is the edge-leakage audit: if gains appear only at smoother settings whose residual correlates with true boundaries, the method is removing anatomy rather than texture, and we will report it that way.
References
- Multi-Centre, Multi-Vendor and Multi-Disease Cardiac Segmentation: The M&Ms Challenge
- AMOS: A Large-Scale Abdominal Multi-Organ Benchmark for Versatile Medical Image Segmentation
- Benchmarking the Robustness of Semantic Segmentation Models
- On the Texture Bias for Few-Shot CNN Segmentation
- FRCNet Frequency and Region Consistency for Semi-supervised Medical Image Segmentation
- DyCON: Dynamic Uncertainty-aware Consistency and Contrastive Learning for Semi-supervised Medical Image Segmentation
- Constructing and Exploring Intermediate Domains in Mixed Domain Semi-supervised Medical Image Segmentation
- Dual-supervised Asymmetric Co-training for Semi-supervised Medical Domain Generalization
- Unsupervised Domain Adaptation through Shape Modeling for Medical Image Segmentation
- Single-Domain Generalization in Medical Image Segmentation via Test-Time Adaptation from Shape Dictionary