Learning the Shape of Scanner Shift: Functional PCA over Intensity Maps for Medical Segmentation

When an MRI segmentation model trained at one hospital is run on another hospital's scanner, accuracy can collapse. Much of the damage comes from how each scanner maps tissue to gray level. A prostate gland that looks mid-gray on one vendor's T2-weighted scan can look brighter, darker, or differently contrasted against fat on another. The strongest domain-generalization method in our material, AdverIN, treats this as a search over arbitrary monotone intensity curves: it trains the network against the worst-case curve inside a bound. Our observation is that real scanner shifts are not arbitrary curves. They are samples from a structured family that can be learned. Each training scan's intensity-to-reference mapping is a smooth function, and functional principal component analysis (FPCA), a method that finds the few dominant "shapes" along which a collection of curves varies, can extract that family from source-site data alone.

We propose to fit FPCA to per-scan, tissue-conditional intensity maps collected across the training sites. The resulting low-dimensional, interpretable model of scanner appearance serves three uses. It drives augmentation, by sampling and extrapolating realistic appearance curves during training. It gives a label-free shift score for a new scan, measuring how far that scan's map lies from the source family. And it supports a constrained input correction that leaves a frozen, certified model untouched. The central question is empirical and falsifiable: does restricting intensity perturbations to a learned functional subspace beat both unconstrained adversarial curves and random smooth curves of matched magnitude?

Core Claim

The claim is that scanner-induced appearance variation in same-sequence MRI concentrates in a handful of functional modes of the intensity map. If so, augmenting along those modes, with deliberate extrapolation beyond the observed range, should generalize to unseen scanners better than perturbations that ignore this structure.

Our method borrows from two prior works. The first is the Karhunen–Loève truncation from Functional Data Analysis, which represents a random curve as a mean curve plus a few weighted basis curves. The second is the monotone-perturbation insight of AdverIN. AdverIN showed that structured monotone intensity attacks help where isotropic noise hurts: Szumel et al. found that Gaussian-noise augmentation fails and even degrades performance under scanner shift. AdverIN's curves are bounded but otherwise unconstrained, and they are chosen adversarially rather than learned from real scanners. We learn the space of curves itself.

The idea also differs from the existing work that connects functional data analysis to domain shift. Functional Optimal Transport uses an FPCA basis to learn a linear operator that transports whole function distributions, and it is tested only on sinusoids and robot trajectories. It forces distributions into alignment, which INPUTADAPTER suggests can be the wrong target: its best adapter actually moved target data further from the source in Wasserstein distance. Contrastive FPCA and Eigen-Adjusted FPCA separate group-specific from shared modes and model covariates through eigenvalues, respectively, but they are used on demographic groups, fMRI, and gait curves. None of these works model scanner appearance, and none connects to segmentation training.

Research Program

Constructing the functional datum. For each labeled source scan $i$ we compute tissue-conditional intensity quantile functions using the ground-truth masks (for example gland, surrounding tissue, background for prostate). Conditioning on tissue removes most of the anatomy-driven variation that would contaminate a global histogram. The amount of fat in the field of view, for instance, changes a global histogram but not the within-tissue quantiles.

We then fit a single monotone map $T_i$ that best aligns these quantiles to a pooled reference. To guarantee that every curve we later generate is monotone, we work with the log-slope $w_i(u) = \log T_i'(u)$, where $u \in [0,1]$ is normalized intensity. The FPCA model is

$$w_i(u) = \mu(u) + \sum_{k=1}^{K} \xi_{ik}\,\phi_k(u),$$

where $\mu$ is the mean log-slope, $\phi_k$ are orthonormal modes, and $\xi_{ik}$ are scan-level scores with variances $\lambda_k$. Integrating $\exp(w)$ recovers a valid monotone map for any score vector.

Separating scanner from patient variation. Scans from the same site still vary for biological reasons such as edema or age-related signal change. As an optional refinement, we form the contrastive operator $C_{\text{between-site}} - \alpha\, C_{\text{within-site}}$ in the style of CFPCA. Choosing $\alpha$, and the truncation level $K$, without target labels is a known open problem in both CFPCA and regularized functional covariate-shift learning. We choose both by leave-one-source-site-out validation within the training sites. We do this with some care, because Deep Neural Network Regression with Functional Covariates shows that too-small $K$ adds bias while too-large $K$ passes noise downstream.

Three uses of the model. Augmentation samples $\xi_k \sim \mathcal{N}(0, \gamma^2 \lambda_k)$ with an inflation factor $\gamma > 1$, so training sees plausible but unseen scanners. We also test an adversarial variant that runs AdverIN's min-max search restricted to the $K$-dimensional score space. The shift score is the Mahalanobis distance of a target scan's estimated scores. On target scans, tissue masks come from the frozen model's own predictions. Frozen-model correction maps a target scan through the monotone map, within the learned span, that best aligns its predicted-tissue quantiles to the reference. We iterate predict-then-correct a small fixed number of times. This last component is the most fragile, because it can confirm its own errors, and we treat it as secondary.

Scope. We restrict the study to shifts within a nominal sequence, such as T2-weighted MRI across vendors. A single monotone map cannot express a change of sequence, such as bSSFP to LGE, and we do not claim it can.

Baselines

The naive reference is a 3D U-Net trained on pooled source sites with standard z-score normalization and default augmentation. We choose the U-Net because AMOS found a well-tuned 3D-UNet beat Swin-UNetr on multi-scanner abdominal CT (89.04% vs 86.32% mean Dice), so it is a fair and strong backbone. On the six-center prostate leave-one-domain-out benchmark, AdverIN reports 79.8% average Dice for its baseline.

The strongest existing competitor is AdverIN itself: 82.4% average Dice on the same prostate benchmark, 74.84% on the hardest domain E (baseline 61.22%), and 81.13% on retinal fundus against an 87.24% target-trained upper bound. It is the right competitor because it attacks exactly the same perturbation class, monotone intensity maps, without learning its structure. We will rerun AdverIN in our codebase with the same backbone, data splits, training epochs, and a hyperparameter search budget equal to ours. Beating it means a higher leave-one-domain-out average Dice with non-overlapping bootstrap confidence intervals, together with a gain on the hardest domain. Matching its average while clearly improving the worst domain would also count as a meaningful result.

The ablation that isolates our key ingredient is random smooth monotone curves. We draw log-slope curves from a generic smooth prior, such as random low-order splines, rescaled so their average perturbation magnitude equals that of our FPCA samples. If this matches FPCA augmentation, the learned structure is doing nothing.

For the frozen-model correction, the competitors are three classical pipelines. The first is N4 bias-field correction plus Nyúl histogram standardization applied to the frozen model's inputs. The second is the same pipeline applied to predicted-tissue histograms, which isolates the learned subspace from the tissue conditioning. The third is post-hoc calibration in the manner of Ouyang et al., included for the calibration metrics only.

Why not just…?

Why not just use histogram standardization and bias-field correction? This is the most serious threat to the frozen-model correction. Nyúl standardization already fits a piecewise monotone map to a reference histogram. We argue that the global histogram mixes anatomy content with scanner response, and that tissue conditioning plus a learned subspace removes that confound. The file offers only indirect evidence. Szumel et al. report an average MRI ΔAUC of −0.097 under standard preprocessing, and AMOS reports drops of over 20% mean Dice on unseen MRI scanners. Neither isolates Nyúl specifically. Our second baseline, tissue-conditional Nyúl without the FPCA subspace, settles the question directly. If it matches our correction, we will report that the correction's value lies in tissue conditioning, not FPCA.

Why not just keep AdverIN's unconstrained worst-case curves? An adversary over all bounded monotone curves may spend capacity on curves no scanner produces, while under-sampling the directions real scanners take. Our learned subspace could, however, be too narrow, especially with only five source sites in the prostate benchmark. The γ-inflation experiments and the head-to-head at matched budget will show which effect dominates. We also test a hybrid, adversarial search in score space, in case the answer is to combine the two.

Why not just use random smooth monotone augmentation? If scanner curves have no special structure beyond smoothness, generic smooth curves of the right magnitude should work equally well. That would make FPCA unnecessary. The magnitude-matched ablation is designed to show this outcome if it is true, and we treat it as the decisive check.

Why not just pool some target-scanner data? Szumel et al. show that mixing in 10–20% target data recovers most of the loss. That remedy requires annotated data from every deployment scanner in advance, which zero-shot deployment rules out. We report it as an upper-bound reference, not a competitor.

Proposed Experiments

Prostate MRI. The main testbed is the six-center T2-weighted prostate MRI leave-one-domain-out benchmark from AdverIN. It is same-sequence and multi-vendor, and it has a documented hard domain. We report Dice and 95th-percentile Hausdorff distance (HD95, where AdverIN reports 7.22 mm) per held-out domain.

AMOS MRI. The second testbed is leave-one-scanner-out on the 100 MRI scans of AMOS, which span multiple scanners. It tests multi-organ tissue conditioning and offers more distinct scanners from which to learn modes. AMOS reports a drop of more than 20% mean Dice on unseen scanners. We found no reported result on Lacuna for per-scanner leave-one-out Dice on AMOS MRI, so we establish that baseline ourselves.

Retinal fundus. Four-center retinal fundus segmentation serves as a stress test. We use multivariate FPCA over the three color channels, and compare against AdverIN's 81.13% average Dice and the 87.24% upper bound.

Decisive checks. The first check is whether FPCA augmentation beats both AdverIN and the magnitude-matched random-curve ablation on the hardest held-out domains. The second is whether the explained-variance curve shows a genuinely low-dimensional structure, for example most of the between-scan variance in two to four modes. If the variance is spread across many modes, the core premise fails and we would report that. The third is whether the shift score correlates with per-scan Dice drop across held-out sites, measured by Spearman correlation against a baseline score based on raw-histogram distance. The fourth is whether frozen-model correction beats tissue-conditional Nyúl, with calibration error reported alongside Dice.

As a sanity check, we visualize the leading modes $\phi_k$. We expect them to read as recognizable physical effects, such as global contrast stretch and bright-end compression, rather than noise.