Source-Estimated Ground Metrics for Few-Scan Scanner Adaptation

Medical segmentation models often fail when moved to a new hospital. The new scanners, field strengths, and protocols shift the image statistics, and the model has never seen them. Optimal transport (OT) gives a principled fix. It finds the cheapest way to move the new hospital's internal features onto the training hospital's features, and it measures "cheap" with a chosen distance called the ground metric. The fix breaks down in the setting where hospitals actually need it, which is when only a handful of unlabeled scans from the new site are available. With five scans and a 256-dimensional feature space, the matching is mostly noise. Learning the ground metric from source and target data together, as current metric-learning OT methods do, makes this worse, because a full metric has far more free parameters than five scans can pin down.

We propose to choose the ground metric before any target scan is seen, using only the training hospital's data. From source images we estimate the few feature directions that change when the same anatomy is imaged differently, using physics-based acquisition perturbations and, where available, real differences between source sites. We then let transport act only along those directions. This does two things. It cuts the effective dimension of the transport problem from hundreds to a handful, so a few target scans can support it. It also stops alignment from erasing real anatomical or pathological differences in the target cohort, such as a higher lesion load, because those differences lie outside the allowed directions. The segmentation network stays frozen. Adaptation is a small correction applied to its internal features, which fits the regulatory constraint against retraining certified models.

Core Claim

When target data are scarce, the ground metric decides whether OT adaptation helps. Specifically, it matters which subspace the metric charges for, and that subspace must come from source-side knowledge of acquisition physics rather than from target data.

Let $U \in \mathbb{R}^{d\times k}$ be an orthonormal basis for the acquisition subspace, where $d$ is the bottleneck feature dimension and $k \ll d$ is its rank. We estimate $U$ from two matrices. The first, $\Sigma_{\text{acq}}$, is the covariance of feature differences between a source patch and its acquisition-perturbed copies. The second, $\Sigma_{\text{anat}}$, is the covariance of features across different source patches. $U$ is spanned by the top $k$ solutions $u$ of the generalized eigenproblem

$$\Sigma_{\text{acq}}\, u = \rho\, \Sigma_{\text{anat}}\, u .$$

These are the directions with the most acquisition variation per unit of anatomical variation. The ground metric is then $c_U(x,y) = \|U^\top(x-y)\|^2$, a rank-$k$ Mahalanobis cost between a target feature $x$ and a source feature $y$. We solve OT in the $k$-dimensional projected space, optionally conditioned on predicted labels in the JDOT style. Each target feature is then moved only within the subspace:

$$\tilde{x} = x + U\big(T(U^\top x) - U^\top x\big),$$

where $T$ is the barycentric transport map computed in the projected space. Empirical Wasserstein estimates converge roughly as $n^{-1/\text{dim}}$; this is the finite-sample term in the bound of Theoretical Analysis of Domain Adaptation with Optimal Transport. Replacing $d$ with $k$ should therefore make the procedure usable at $n \le 10$ target scans, where full-dimensional alignment is not.

We are not claiming that a nonlinear transport map is needed inside the subspace. A Gaussian (mean and covariance) map restricted to $U$ is one of our variants. If it matches full OT within $U$, then the contribution is entirely the choice of metric, and we will report it that way.

Relation to Prior Work

Three groups of papers bear on this, and they play different roles.

The first group motivates the gap. Seg-JDOT aligns 3D-UNet bottleneck features and predicted labels across MS-lesion MRI sites with a fixed squared Euclidean cost. Its authors tried a Dice-based cost and found it too noisy for the solver. They relied on dense patch sampling from full site cohorts and never tested a few-scan regime. Multi-center benchmarks show the problem is real and unsolved. AMOS reports a drop of over 20% mDice on unseen MRI scanners. FeTA 2024 attributes about 0.075 Dice to acquisition site even for heavily augmented nnU-Net pipelines. We keep Seg-JDOT's frozen-feature alignment and replace its isotropic cost.

The second group supplies or constrains the method. MLOT jointly learns a Mahalanobis metric and transport plan on Office-Caltech, and it shows that PCA is the optimal linear projection for Wasserstein distance to a subspace. A Riemannian Approach to Ground Metric Learning learns a full SPD (symmetric positive definite) metric in closed form. Its weighted covariance becomes rank-deficient when $d > n$, which forces ad hoc shrinkage, and its result depends on an unchosen reference matrix. Both learn the metric from the adaptation data itself. We learn nothing from target data except the $k$-dimensional transport. In effect, our $\Sigma_{\text{anat}}^{-1}$-whitened acquisition basis is a principled answer to the open question of how to pick that reference matrix.

A Swiss Army Knife for Minimax Optimal Transport recovers low-dimensional signal subspaces by maximizing transport cost over a class of costs. With five target scans, the subspace that maximizes discrepancy will tend to be sampling noise. Our subspace is fixed by source-side physics, which is the opposite way of choosing it.

OT-SI learns a neural cost from group-level correspondences and reaches 56.1% versus 9.9% for Euclidean OT on RNA-to-protein alignment. We borrow its idea that weak correspondence can shape the cost. Our correspondences, a patch and its perturbed copies, are generated on the source side, so no target-side groups are needed.

The third group is a tension we test directly. INPUTADAPTER adapts inputs for frozen clinical time-series predictors. It found that its successful adapter moved target data further from the source in Wasserstein distance: it removed artifacts rather than matching distributions. Restricting transport to acquisition directions is our proposed reconciliation. The method should remove the artifact component and leave non-acquisition divergence untouched. INPUTADAPTER uses target labels and time series, whereas we use no target labels and volumetric images.

Ouyang et al. recalibrate frozen MRI segmenters after the fact and do not change Dice. Their test-time-augmentation susceptibility signal is close in spirit to our $\Sigma_{\text{acq}}$, but we use it to correct features rather than confidence.

Research Program

The source model is a 3D U-Net trained with standard nnU-Net-style augmentation and then frozen. We also run a no-augmentation variant to see how much residual acquisition sensitivity augmentation leaves.

To estimate the subspace, we generate perturbed copies of every source patch. The perturbations cover bias fields, gamma and contrast curves, noise level, blur and resolution changes, and motion and k-space artifacts. We record bottleneck features for each copy, form $\Sigma_{\text{acq}}$ and $\Sigma_{\text{anat}}$, and solve the generalized eigenproblem. When the source has more than one site, as with the two FeTA training sites, we add the real between-site covariance of class-conditional feature means to $\Sigma_{\text{acq}}$. The rank $k$ is chosen on source sites only. One source site is held out as a pseudo-target, and we pick the $k$ that best adapts it from five of its scans.

At deployment, we collect bottleneck features from $n$ unlabeled target scans. We compute entropic OT in the $k$-dimensional projection, class-conditioned on the frozen model's predictions, and apply the subspace-restricted map. The corrected features then pass through the frozen decoder.

We will also analyze two diagnostics. The first is how much of the real cross-site feature displacement falls inside $U$, measured as the captured fraction of the squared norm of the mean shift. The second is how adaptation error scales with $n$ and $k$, to check against the predicted $n^{-1/k}$ behavior.

Baselines

The naive reference points are the frozen source model applied to target scans with only per-volume z-scoring, and the same model with histogram matching to the source. These show how much any adaptation adds. Seg-JDOT reports its gains as a relative improvement of about 429% Dice over non-adapted on Site 08 to Site 07. We will reproduce the absolute non-adapted numbers ourselves.

The strongest existing method is Seg-JDOT, full-dimensional squared-Euclidean joint OT at the bottleneck. It is the only medical OT method in the file and the one our metric directly replaces. Its only reported figure is the relative gain of about 429% Dice on Site 08 to Site 07. We found no result on Lacuna for Seg-JDOT with fewer than 20 target scans. For fairness, we run it at inference time on the same frozen network, with the same patch sampling, the same entropic solver, and the same predicted-label conditioning. The ground metric is the only difference. We will also run its original variant that fine-tunes the encoder, as an upper reference that is not weight-frozen.

The strongest learned-metric competitors are MLOT and the Riemannian SPD metric, each fit on source and target bottleneck features. They get the same shrinkage and tuning grid, chosen on the source pseudo-target split. Minimax OT with a Schatten-norm Mahalanobis class is also included, with its rank matched to our $k$. No reported result was found on Lacuna for any of these on medical images.

A simple test-time baseline matches feature means and variances, or full covariances (Gaussian OT), in all $d$ dimensions.

The ablations that isolate the key ingredient keep everything fixed and swap only the subspace $U$, at equal rank $k$. The alternatives are a random subspace, the top-$k$ source PCA directions (MLOT's projection), the top-$k$ discrepancy directions estimated from the target scans, and our acquisition subspace.

Beating the baselines means two things. Our method must give higher mean Dice and lower HD95 than full-dimensional Seg-JDOT and the learned metrics at $n \le 5$, with paired per-subject tests across target sites. It must also beat the PCA and target-discrepancy subspaces at equal $k$. For FeTA's 0.55T set, where the challenge reports Dice 0.86, HD95 1.69, and ED 6.26 for the best entries, our frozen model plus adaptation should narrow the gap between our own unadapted model and in-domain performance without worsening ED.

Why not just…?

Why not just use moment matching or Gaussian OT in the full feature space? This is the most serious threat, because with few scans the first two moments are what is estimable. A full $d\times d$ covariance from five scans' patches is still badly conditioned, and full-dimensional mean matching also shifts anatomy-related directions. Our ablation runs Gaussian OT both in full dimension and inside $U$. If Gaussian OT inside $U$ is as good as full OT inside $U$, we will report the method as subspace-restricted moment matching. The claim survives as long as the restriction to $U$ beats the full-dimensional version.

Why not just rely on heavier augmentation during training? FeTA 2024 shows that top teams already used extensive augmentation, yet site and image quality still explained about 0.075 and 0.10 Dice. Augmentation leaves residual sensitivity, and our $\Sigma_{\text{acq}}$ is measured on the augmented model to target that residual. If augmentation fully removed sensitivity, $\Sigma_{\text{acq}}$ would be near zero and our correction would do nothing. The augmented and non-augmented source-model arms will show this directly.

Why not just learn the ground metric from source and target, as MLOT or the Riemannian method do? At large $n$ that may well be better, and we expect the curves to cross. At $n \le 10$, the Riemannian method's own rank-deficiency issue when $d > n$ suggests the learned metric will fit sampling noise. The Dice-versus-$n$ curves are the decisive test. If learned metrics match ours at $n = 5$, our central claim is wrong.

Why not just fine-tune on a few labeled target scans? This needs expert labels and voids clearance of a locked model, so it answers a different question. We still report it at $n = 1$ and $n = 5$ labeled scans as a ceiling, which tells us how much of the labeled-adaptation gain label-free subspace transport recovers.

Proposed Experiments

We use three benchmarks from the file. The first is the MICCAI 2016 MS lesion data, with four centers and Seg-JDOT's site pairs, evaluated with Dice, HD95, and lesion-wise detection. The second is FeTA 2024, training on KISPI and Vienna and testing on out-of-domain and KCL 0.55T data, evaluated with Dice, HD95, and Euler characteristic difference. The third is AMOS MRI across scanners not seen in training, evaluated with mDice and with small organs reported separately. For every method, target cohort size varies over $n \in \{1, 2, 5, 10, 20\}$ unlabeled scans, using 20 random draws per $n$.

Four checks decide the proposal. The first is the scaling check. Our advantage over full-dimensional and learned-metric OT should be largest at $n \le 5$ and should shrink as $n$ grows. A flat or reversed pattern falsifies the sample-complexity argument.

The second is the subspace check at equal $k$. The acquisition subspace must beat the random, source-PCA, and target-discrepancy subspaces. This is what shows that source physics, not merely low rank, carries the benefit.

The third is the coverage diagnostic. On held-out source sites and the FeTA training sites, we measure what fraction of the real cross-site mean feature shift lies inside $U$. If simulated perturbations capture little of the real 0.55T shift, the method has a stated failure mode and we report it rather than tune around it.

The fourth is the pathology-preservation test, which addresses the INPUTADAPTER tension. On MS data, we build target cohorts with deliberately high or low lesion load. We then measure bias in predicted total lesion volume after adaptation, along with how much Wasserstein mass each method moves. Full-dimensional alignment should pull lesion volume toward the source cohort's load. Our method should move less mass and leave volume estimates unbiased while still improving Dice.