Segmentation networks for medical scans often fail when moved to a scanner from a manufacturer they never saw in training. Classical low-rank methods offer a tempting fix. Decompose a feature matrix into independent directions, ranked by how much energy each carries (its singular values). Then shrink the weak directions hard and leave the strong ones alone. This "adaptive singular value weighting" works well for image denoising, where weak directions really are noise. The opposite heuristic, though, works for domain adaptation: there the strongest direction is the one that soaks up domain differences, and it gets penalized. In MRI, scanner differences in gain, contrast, and coil bias are global and high-energy. So there is good reason to doubt that vendor information sits in the weak tail of the spectrum.
We propose to stop guessing. Before training, we measure, direction by direction, how much each singular direction of a segmentation network's features varies across scanner vendors. The penalty on each singular value then scales with that vendor sensitivity rather than with its size. In plain terms, the network is taught to quiet the feature directions that tell scanners apart, whether those directions are large or small, while keeping the directions that carry anatomy. The method needs only vendor tags, which come free with every DICOM header. It also uses unlabeled scans from extra vendors, which the M&Ms cardiac MRI protocol already provides. The test is whether it closes the remaining gap on a completely unseen vendor, on top of the intensity augmentation that won that challenge.
Core Claim
Two lines of work disagree about which end of the spectrum holds the nuisance. Weighted Nuclear Norm Minimization shrinks each singular value by an amount inversely proportional to its size and gets closed-form optimal denoising. SoMA builds on the same intuition: it freezes the principal components of pretrained weights as robust "world knowledge" and adapts only the minor components. Batch Spectral Penalization (BSP) finds the reverse under adversarial alignment, where the top singular value inflates and absorbs domain-fooling signal, so it penalizes the top few singular values. These two lines motivate the proposal: each is right in its own regime, and neither checks the regime it is in.
Our claim is that for hardware-induced shift in medical images, the magnitude of a singular value is a poor proxy for whether it encodes the scanner. A per-direction vendor-sensitivity score is a better basis for the weights. Concretely, let $F \in \mathbb{R}^{N \times d}$ be a mini-batch of decoder-input features, with $N$ spatial positions pooled across scans and $d$ channels. Let its singular value decomposition be $F = U \Sigma V^\top$, with singular values $\sigma_i$ and right singular vectors $v_i$. Let $p_i = F v_i$ be the projections of all positions onto direction $i$. We define the vendor sensitivity as the share of the variance of $p_i$ explained by vendor identity:
$$s_i = \frac{\mathrm{Var}_{\text{between vendors}}(p_i)}{\mathrm{Var}_{\text{total}}(p_i)} \in [0,1].$$
This is the variance-ratio idea of Verbanck et al., who shrink each principal dimension by its signal-to-total variance ratio. We turn it around so that "signal" means scanner identity. The training loss adds
$$L_{\text{spec}} = \lambda \sum_i \operatorname{sg}(s_i)^{\gamma}\, \sigma_i ,$$
where $\operatorname{sg}$ is stop-gradient, $\lambda$ sets overall strength, and $\gamma$ sharpens the weighting. Recomputing the weights from the current features at each step, with gradients blocked through the weights, follows the iterative reweighting logic of ISTRA. It also follows the two-stage adaptive scheme of Bach et al., whose rank-consistency guarantee depends on weights built from a preliminary estimate rather than optimized jointly. The gradient of $\sigma_i$ with respect to $F$ is $u_i v_i^\top$. That is stable even when singular values nearly tie, unlike gradients through singular vectors. With $d$ around 320 at an nnU-Net bottleneck, the decomposition costs far less than one forward pass. This sidesteps the full-SVD cost that limits adaptive weighting in deep networks.
Research Program
The work has three parts, and the first can falsify the rest.
The first part is a spectral diagnosis of where vendor information lives. We train nnU-Net on M&Ms vendors A and B, with and without histogram-matching augmentation. At several depths we compute $s_i$ as a function of singular value rank on held-out scans from vendors A, B, C, and the unseen vendor D. Prior evidence that scanners leave footprints even in raw signals (Cole et al.) suggests vendor information will be present. The open question is where. There are three possible outcomes. If sensitivity concentrates in the tail, magnitude-based weighting in the style of WNNM is the right tool and our score adds little. If it concentrates in the head, BSP's intuition carries over to segmentation. If it is spread unevenly, which is our hypothesis, neither magnitude rule can target it. Each outcome is publishable, and the diagnosis also tells us whether augmentation has already removed vendor information from the features.
The second part is the regularizer itself. Vendors A, B, and C enter the sensitivity estimate, and C contributes no segmentation loss, matching the official protocol in which C is unannotated. Mini-batches are vendor-balanced so that between-vendor variance is estimable. One risk is that the network simply rotates vendor information into directions that are not yet penalized. Recomputing $s_i$ every step limits this, and we monitor it directly with a linear vendor probe trained on frozen features after training.
The third part guards against over-cleaning. In M&Ms, pathology is partly confounded with center and therefore with vendor. A regularizer that removes vendor-predictive directions could also remove disease-relevant shape. This is the concern that led Degel et al. to add an anatomical shape prior next to their vendor adversary. We therefore compute $s_i$ as a partial variance ratio: between-vendor variance after first regressing out the disease label, which is available for training subjects. We report a linear disease probe alongside the vendor probe.
Baselines
The naive reference point is nnU-Net trained on vendors A and B with default augmentation. The M&Ms report Multi-Centre, Multi-Vendor and Multi-Disease Cardiac Segmentation describes a baseline drop of about 6% on unseen hardware for this kind of model.
The strongest existing competitor is nnU-Net with histogram-matching augmentation (Histogram Matching Augmentation for Domain Adaptation). It placed third in M&Ms with hidden-test Dice of 0.9051 for the left ventricle, 0.8405 for the myocardium, and 0.8749 for the right ventricle. The challenge report found that intensity augmentation of this kind beat explicit domain-alignment networks, and its best method reduced the unseen-vendor gap to 1.5%. We take this as the bar. Every spectral method in this study runs on top of histogram matching and the full nnU-Net augmentation pipeline, with the same splits, the same 1000-epoch schedule, five seeds, and an equal hyperparameter search of twelve configurations per method on a vendor-A/B validation fold. Vendor D is never used for tuning. Beating this baseline means a gain in mean vendor-D Dice and a drop in vendor-D 95th-percentile Hausdorff distance that both exceed twice the seed standard deviation, with no loss on vendors A and B.
The spectral competitors share the same backbone and training. The first is uniform nuclear-norm shrinkage of the same features. The second is magnitude-weighted shrinkage with $w_i \propto 1/(\sigma_i+\epsilon)$, following Gu et al.. The third is BSP's top-$k$ penalty. The fourth is the polarization loss of BSP-WSA, which boosts large and suppresses small singular values; we adapt it from prediction matrices to feature matrices. The fifth is linear-dependency regularization, LDDG, which imposes a hard rank target on cross-domain class features. LDDG is the closest medical-imaging precedent. Its key differences from our method are that it imposes a discrete rank target with no notion of which directions carry the scanner.
Two ablations isolate the key ingredient. In the first, we keep everything but shuffle the vendor-sensitivity scores across directions, so the loss keeps the same total weight but loses the targeting. In the second, we replace the soft penalty with hard removal: the top vendor-sensitive directions are projected out, with a cutoff chosen on validation.
Why not just…?
Why not just use stronger intensity augmentation? This is the most serious threat. The M&Ms evidence says augmentation beat every alignment method the challenge saw. We do not argue around it. Every experiment sits on top of histogram matching, and the diagnosis stage measures how much vendor sensitivity survives augmentation. If vendor information in the features is near zero after histogram matching, the regularizer has nothing to act on, and we will report that augmentation already solves the feature-level problem. The histogram-matching author also warns that intensity remapping can erase diagnostic contrast, such as scar in late gadolinium enhancement. A feature-level method avoids altering inputs, but that advantage only matters if it also improves accuracy.
Why not just add a vendor-classifier adversary? Degel et al. did this for 3D echocardiography and needed a shape autoencoder to keep anatomy intact. BSP showed that adversarial alignment concentrates energy in the top singular value and erodes discriminability. M&Ms found that adversarial entries trailed augmentation. Our loss involves no minimax game and acts on identified directions. We still include a DANN-style adversary on the same features as an additional baseline, so that the comparison is direct.
Why not just project out the vendor directions once, or use a SoMA-style hard split? Projecting out a linearly discriminant vendor subspace is simpler and could plausibly match our method. That is exactly why it is our hard-removal ablation. SoMA's own setting differs from ours. It assumes a pretrained foundation backbone whose principal components are trustworthy, and it splits them with a manual rank cutoff. The diagnosis stage tests whether "principal equals robust" holds in from-scratch medical segmentation features. If hard removal matches the soft penalty, we will report that the simpler method suffices.
Why not just use magnitude-based WNNM weighting? If vendor sensitivity turns out to sit in the spectral tail, it will work, and the shuffled-score and magnitude-weighted runs will show it. The proposal is worth pursuing only if the diagnosis shows sensitivity and magnitude decoupled.
Proposed Experiments
The primary benchmark is M&Ms. We train on annotated vendors A and B, include unannotated vendor C only in the sensitivity estimate, and test zero-shot on Canon vendor D. We report per-structure Dice and Hausdorff distance per vendor, and the seen-to-unseen gap. As a second hardware-shift setting we use the four-center spinal cord gray matter MRI data. There, LDDG reports Dice of 0.7881 against a 0.7425 baseline, and we run leave-one-center-out with the same protocol.
If access allows, we add the dual-scanner retinal OCT data (Zeiss Cirrus versus Heidelberg Spectralis) used by Latent Metric Schrödinger Bridges, training layer segmentation on one device and testing on the other. That paper reports harmonization boundary error of 0.43 pixels against 0.89 for a diffusion baseline. Those figures measure image harmonization, not segmentation, so they are context rather than a target, and no reported cross-device segmentation result was found on Lacuna for that dataset. Because the regularizer runs at training time, it does not compete with harmonization; we test whether it composes with it.
The decisive checks follow from the claim. The first is the diagnostic: the correlation between $s_i$ and singular value rank must be weak for the method to have a reason to exist. Second, on vendor D, the vendor-sensitivity penalty must beat histogram matching alone, the magnitude-weighted variant, BSP, and the shuffled-score ablation by more than twice the seed standard deviation. Third, the post-hoc vendor probe accuracy must fall while the disease probe accuracy stays within one point of baseline. Without that, a Dice gain could come from removing pathology cues that happen to correlate with the test vendor. If the first check fails, the paper becomes a spectral characterization of scanner shift. If the second fails while the third succeeds, we learn that removing vendor information from features is not what limits unseen-vendor accuracy. That would be a useful negative result for the whole invariance line of work, including the 20%-plus unseen-scanner drops reported by AMOS.
References
- Weighted Nuclear Norm Minimization with Application to Image Denoising
- SoMA: Singular Value Decomposed Minor Components Adaptation for Domain Generalizable Representation Learning
- Transferability vs. Discriminability: Batch Spectral Penalization for Adversarial Domain Adaptation
- Regularised PCA to denoise and visualise data
- A Nonconvex Relaxation Approach for Rank Minimization Problems
- Consistency of Trace Norm Minimization
- Predicting brain age with deep learning from raw imaging data results in a reliable and heritable biomarker
- Domain and Geometry Agnostic CNNs for Left Atrium Segmentation in 3D Ultrasound
- Multi-Centre, Multi-Vendor and Multi-Disease Cardiac Segmentation: The M&Ms Challenge
- Histogram Matching Augmentation for Domain Adaptation with Application to Multi-centre, Multi-vendor and Multi-disease Cardiac Image Segmentation
- Batch Singular Value Polarization and Weighted Semantic Augmentation for Universal Domain Adaptation
- Domain Generalization for Medical Imaging Classification with Linear-Dependency Regularization
- Optical Coherence Tomography Harmonization with Anatomy-Guided Latent Metric Schrödinger Bridges
- AMOS: A Large-Scale Abdominal Multi-Organ Benchmark for Versatile Medical Image Segmentation