A segmentation network cleared as a medical device cannot have its weights changed after deployment. When a hospital installs a scanner from a vendor the network never saw, accuracy can collapse. The AMOS benchmark reports drops of over 20% mean Dice on MRI from unseen scanners. In the M&Ms cardiac challenge, the unseen Canon vendor stayed the hardest case for every team. We propose a small, auditable fix applied to the input. Take every scan a new site produces and summarize each one by its intensity profile, meaning its sorted log-intensity values. Stack these profiles into a matrix with one column per scan. Then split the matrix with robust principal component analysis, a convex method that separates a low-rank part from a sparse part. The low-rank part captures the intensity mapping the site's scanner applies to every patient. The sparse part captures scan-specific exceptions such as large scars, unusual fields of view and artifacts. Each scan is then corrected with one global monotone intensity map estimated for the site, and the locked network sees the corrected scan.
The point of the design is where the matrix is built. The obvious transplant of robust PCA would split each image into "low-rank anatomy" plus "sparse scanner corruption." That gets the physics backwards. Scanner effects such as gain, contrast curves and coil-sensitivity falloff are dense and smooth, not sparse. Anatomy is the detailed, high-rank part of an image. The structure that robust PCA actually fits lives across patients at one site: all of them pass through the same scanner transfer function.
Core Claim
Scanner-induced intensity shift at a site is shared across all of that site's scans, so it should be estimated from the site's whole batch rather than scan by scan. Estimating it with a convex low-rank-plus-sparse decomposition has two advantages. It is robust to the minority of scans whose intensity distribution is atypical, which are exactly the scans that fool per-scan histogram matching. It also yields an identifiability diagnostic that tells us when to refuse to harmonize.
Concretely, a target site has $n$ scans. Let $Q \in \mathbb{R}^{K \times n}$ hold, in column $j$, the log-intensities of scan $j$ at $K$ fixed quantile levels inside an automatic body mask. If the scanner applies a gain-and-gamma curve, which in log space is an affine map $x \mapsto a + b x$, then
$$Q = L + S + Z, \qquad \operatorname{rank}(L) \le r + 1.$$
Here $L$ holds the site-mapped profiles of "typical" patients, whose tissue proportions vary in about $r$ modes. $S$ is sparse, holding the quantile entries that individual scans move through pathology or artifacts. $Z$ is small dense noise. We estimate $L$ and $S$ with the noise-tolerant version of Principal Component Pursuit:
$$\min_{L,S}\ \|L\|_* + \lambda \|S\|_1 + \tfrac{1}{2\tau}\|Q - L - S\|_F^2.$$
In this objective, $\|L\|_*$ is the sum of singular values, $\|S\|_1$ is the sum of absolute entries, $\lambda$ trades the two off, and $\tau$ sets the noise tolerance. We then fit the site's monotone map by aligning the column space of $L$ to a stored source template. The template is a mean log-quantile profile plus its principal subspace, kept as aggregate statistics, never as images. The correction applied to every scan is a two-parameter (later, few-knot spline) monotone intensity map. It cannot create or move structures, and a regulator can read it off as a curve.
Prior Work and How This Differs
The method comes from the convex decomposition literature. Wright et al. and Minasyan et al. show that Principal Component Pursuit recovers low-rank and sparse components exactly under incoherence, with a near-universal $\lambda = 1/\sqrt{m}$. We reuse that solver on a small $K \times n$ matrix, where repeated full SVDs cost nothing. Parrilo et al. give the deterministic identifiability condition: sparse supports must not pile up in a few rows or columns. We turn that condition into a deployment safety check. A scan whose deviation fills its entire column is a whole-scan outlier, such as a different sequence, and violates sparse incoherence. We flag it rather than trust the decomposition.
Rambhatla et al. extend the model to dictionary-sparse corruptions. We do not need that extension, because our sparse part is already sparse in the quantile coordinates.
Three medical works motivate the gap without filling it. StyCona applies per-image SVD to treat singular values as "style," but only as training-time augmentation. Its finding that perturbing too many rank-one components corrupts labels shows why per-image spectral splits entangle anatomy and appearance. That finding is what led us to the across-patient construction. ERM-NU penalizes the nuclear norm of learned features and was tested only on natural-image benchmarks. It also requires retraining, which a locked model forbids. FedIAT separates covariate from concept shift with low-rank weight updates over federated rounds, which is likewise incompatible with locking.
The test-time competitors are OptTTA, which searches augmentation policies per target to match frozen batch-norm statistics and minimize entropy, and INPUTADAPTER, which trains a neural input transformer and needs target labels. Both adapt inputs, but through unconstrained optimization with no account of which scans are untrustworthy. Our transform is site-level, convex and globally monotone, and it comes with an abstention signal.
The M&Ms challenge constrains the proposal. It found that intensity augmentation on a well-tuned nnU-Net was the single most effective intervention. That result supports the premise that vendor shift is largely an intensity-mapping problem. It also raises the bar, because a heavily augmented model may already be invariant to what we correct.
Research Program
The work has three parts.
The first part tests the structural premise. On source vendors A and B of M&Ms, we apply known synthetic gain and gamma curves plus injected sparse deviations, such as simulated scar hyperintensity and field-of-view crops. We then measure whether $(a, b)$ is recovered as the deviation fraction grows. We also measure where recovery breaks, comparing that point qualitatively with the random-model phase transition of roughly 0.35 that Wright et al. report for 200×200 matrices. We do not expect the same number, since our matrices are thin and structured. On real data we measure the singular value decay of $Q$ per vendor, to check whether a rank of order 3 to 5 really captures typical patients.
The second part builds the harmonizer. It has three pieces: stable PCP on $Q$; alignment of the column space to the source template, by fitting an affine map and then a monotone spline with few knots; and per-scan column-support statistics of $S$ as the abstention signal.
The third part evaluates the harmonizer in front of locked segmenters on held-out vendors and scanners. It includes ablations that remove, one at a time, the nuclear norm, the sparse term, and the site-level pooling.
Baselines
The naive reference is the unadapted locked nnU-Net. We train it on M&Ms vendors A and B with nnU-Net's default augmentation and evaluate it unchanged on vendors C and D. M&Ms reports a baseline drop of about 6% on unseen vendors. Every method must beat this reference.
The strongest training-side competitor is the same nnU-Net trained with the heavy intensity augmentation used by top M&Ms teams: histogram matching, gamma shifts, noise and contrast changes. The challenge reports this narrowed the cross-vendor gap to 1.5%, with left-ventricle Dice of about 0.88 to 0.92 across vendors for the winner. We run our harmonizer in front of both locked models. Beating this baseline means a statistically significant gain on vendor D, measured by Dice and by worst-decile scan Dice, when the harmonizer is placed in front of the augmented model. Gains only on the weakly augmented model would show that the method duplicates augmentation.
The strongest test-time competitor is OptTTA, which reports a 22.5% Dice improvement over the unadapted source model on M&Ms. We run its public code (OptTTA codebase) on identical locked models and target batches, using its default policy-search budget. We report wall-clock time beside accuracy. Beating OptTTA means matching or exceeding its mean Dice on vendor D at lower variance across scans. A tie in mean with clearly better tail behavior and a working abstention signal would still be a meaningful result, because the transform is auditable.
Three ablations isolate the key ingredients, all using the same template, masks and spline family. Per-scan histogram matching to the source template removes the site-level pooling. A site-level median of quantile profiles keeps pooling and robustness but drops the low-rank model. Plain PCA keeps the low-rank model but drops the sparse term. A further ablation replaces PCP with $\ell_1$ regression onto the fixed source subspace, which tests whether estimating the target subspace with the nuclear norm earns its place.
Why not just…?
Why not just train with heavy intensity augmentation? This is the most serious threat. The M&Ms evidence shows augmentation closes most of the cross-vendor gap, though vendor D remained the weakest for every team. Augmentation fixes the model's invariance only over the distribution of transforms seen during training, and an unseen vendor's curve may fall outside it. The experiment that settles this is the harmonizer placed in front of the augmented model on vendor D and on AMOS unseen MRI scanners. If it adds nothing there, we will report that the method is only useful for models trained without such augmentation.
Why not just histogram-match each scan, or take a robust median over the site? Per-scan matching assumes each patient has the source's tissue proportions. A scan with a large infarct or a different field of view is therefore mapped wrongly, and that error appears as poor worst-scan Dice rather than poor mean Dice. A site-level median repairs outliers but assumes all typical patients share one profile. The low-rank model relaxes that assumption to a few modes of variation. Both are run as ablations. If the median matches PCP, the nuclear norm is not doing anything, and we will say so.
Why not just use OptTTA or batch-norm updates such as TENT? OptTTA already outperforms TENT on these benchmarks according to its own report, so it is the right comparison. Its weakness is not accuracy but transparency. It searches a policy space with entropy and batch-norm objectives that can be satisfied by wrong outputs, and it offers no per-scan reliability signal. The head-to-head comparison and the abstention analysis below show whether our constrained transform gives up accuracy for that transparency.
Why not just translate images with a CycleGAN-style network? In M&Ms, generative vendor synthesis did not reach top ranks. Unconstrained generators can also alter subtle anatomy, which a monotone global intensity map cannot do by construction.
Proposed Experiments
The primary benchmark is M&Ms. We train on vendors A and B, tune on vendor C using only unlabeled data to set the noise level $\tau$, and report Dice and 95th-percentile Hausdorff distance per structure for vendors C and D. AMOS MRI, split by scanner model, is the second benchmark. Its reported drop of over 20% mean Dice on unseen MRI scanners is the gap to close, using a locked 3D U-Net, which AMOS found strongest on CT at 89.04% mean Dice. We will need to verify that per-scan scanner metadata is available to define the splits.
MS-CMR bSSFP→LGE is a deliberate falsification case. Late gadolinium enhancement reverses tissue contrast relationships, so a single monotone site map should fail there. StyCona's 73.39% Dice on that transfer serves as a reference that we expect not to beat. A clear failure on this case, with gains on vendor shifts, would show that we have characterized the method's scope correctly.
Three decisive checks remain. The first is the gain on vendor D in front of the heavily augmented model. The second is whether the column-support energy of $S$ predicts low-Dice scans, scored with the area under the ROC curve for flagging scans whose Dice falls below 0.7. The third is site batch size: how few scans (5, 10, 20) a site needs before the site-level estimate beats per-scan matching. That answers when a hospital could switch the method on.
References
- AMOS: A Large-Scale Abdominal Multi-Organ Benchmark for Versatile Medical Image Segmentation
- Multi-Centre, Multi-Vendor and Multi-Disease Cardiac Segmentation: The M&Ms Challenge
- Robust Principal Component Analysis: Exact Recovery of Corrupted Low-Rank Matrices via Convex Optimization
- Robust principal component analysis?
- Rank-Sparsity Incoherence for Matrix Decomposition
- A Dictionary Based Generalization of Robust PCA
- Style Content Decomposition-based Data Augmentation for Domain Generalizable Medical Image Segmentation
- Domain Generalization via Nuclear Norm Regularization
- Shift-Dependent Asymmetry: Orthogonal Inverse Low-Rank Adaptation for Federated Medical Segmentation
- OptTTA: Learnable Test-Time Augmentation for Source-Free Medical Image Segmentation Under Domain Shift
- Adapt the Data, Not the Model: Input-Space Adaptation for Frozen Time-Series Predictors