Is the Stanford Slump a Contrast Curve? Few-Scan Input Adaptation for Isointense Infant MRI

Six-month-old infant brains are among the hardest structures in medical imaging to segment. Myelination is half-finished at this age, so gray matter and white matter give almost the same signal on both T1- and T2-weighted MRI. Neural networks still separate them well when trained and tested on one scanner. They fail when the scanner vendor changes. In the iSeg-2019 challenge, every top algorithm dropped from Dice 0.90–0.92 on same-site validation to 0.82–0.86 on multi-site test data, and the worst collapse came on GE scans from Stanford after training on Siemens (Multi-Site Infant Brain Segmentation Algorithms: The iSeg-2019 Challenge). Histogram matching and tissue-wise intensity augmentation did not prevent it. Hospitals that scan infants rarely have more than a handful of annotated, or even unannotated, local scans with which to fix the problem.

We propose to test a specific explanation and turn it into a cheap fix. Our hypothesis is that, near zero contrast, a vendor change acts mostly as a smooth but nonlinear distortion of the joint (T1, T2) intensity relationship. Such a distortion can flip the sign of the tiny gray/white difference that a trained network relies on. If this is true, a segmenter trained at one site can be kept frozen. In front of it we fit a tiny input transform, a few dozen parameters, using one to three unlabeled scans from the new site. That transform is chosen so the frozen model's output has the anatomy-level statistics seen in training: tissue proportions, cortical-shell thickness, and confident predictions. The fit needs no target labels, cannot forget the source site, and has too few degrees of freedom to overfit a single scan's quirks.

Core Claim

We make two claims and separate them on purpose. The scientific claim is about the cause of the slump. It holds that most of the cross-vendor Dice gap on isointense infant MRI can be recovered by some member of a low-dimensional family of joint intensity maps. We can check this with an oracle that fits the map using target labels, which gives an upper bound for the whole family. If even the oracle recovers little, the slump comes from noise texture, resolution, or reconstruction differences, and intensity-space fixes of any kind are the wrong tool. That would be a useful negative result, since iSeg-2019 participants kept trying intensity fixes.

The methodological claim is narrower. If the family is expressive enough, its parameters can be found label-free from one to three target scans using anatomical statistics. This should match or beat label-based one-shot weight adaptation of the kind DuMeta performs, and it should do so without degrading the source site.

Research Program

The source model is a 3D segmenter trained on the 23 iSeg-2019 UNC/Siemens subjects. We use the DU-Net with mixed-scale asymmetric convolutions from 3D-MASNet, the top-ranked iSeg-2019 system, so that we start from a strong model rather than a weak one we can easily beat. Its weights are frozen after training.

The input transform $g_\phi$ has three parts. The first is a monotone piecewise-linear curve per modality, with about 16 knots each. The second is a low-rank coupling term that lets the T2 correction depend on T1 intensity, because the two signals cross differently during myelination. The third is a low-order polynomial bias field per modality, to absorb coil-profile differences. In total, $\phi$ has on the order of 50–80 parameters. The family is deliberately small. Adaptation is in input space, an idea borrowed from frozen-predictor work (Adapt the Data, Not the Model, which transforms target inputs so a regulator-frozen time-series model can stay fixed). Our contribution is to give that idea a family matched to MRI physics and a label-free objective drawn from anatomy.

The label-free objective, fitted on target scans $x_1,\dots,x_k$ with $k \le 3$, is

$$ \phi^\star = \arg\min_\phi \; \frac{1}{k}\sum_{j=1}^{k} \Big[ H\big(f(g_\phi(x_j))\big) + \lambda_1\, D\big(t(f(g_\phi(x_j))),\, t^{\text{src}}\big) + \lambda_2\, D\big(v(f(g_\phi(x_j))),\, v^{\text{src}}\big) \Big] + \lambda_3 \lVert \phi - \phi_0 \rVert^2 . $$

Here $f$ is the frozen segmenter. $H$ is mean voxel-wise prediction entropy. $t(\cdot)$ is the histogram of a differentiable cortical-thickness proxy, computed as soft gray-matter volume divided by soft gray/white interface area within local windows. $v(\cdot)$ is the vector of CSF/GM/WM volume fractions. The superscript "src" marks the same quantities measured on source training labels. $D$ is a Wasserstein-1 distance, and $\phi_0$ is the identity transform.

The thickness term follows 4D Segmentation of Brain MR Images with Constrained Cortical Thickness Variation, which showed in a classical energy-minimization setting that bounding cortical thickness stops unrealistic boundary shifts. iSeg-2019 explicitly noted that no deep entry used such priors. Here the prior acts only as a fitting signal for the tiny transform, never as a loss on network weights. The entropy term alone is known to have degenerate minima, for example mapping everything to CSF. The volume and thickness terms are there to block those, and the ablations test whether they actually do.

The analysis program has three parts. The first is the oracle fit, which gives the ceiling of the transform family. The second compares the label-free fit to that ceiling as $k$ goes from 1 to 3. The third asks where the remaining error sits, measuring it by region in the straight and lingual gyri, where iSeg-2019 found errors concentrate. If residual error stays in those narrow folds after adaptation, it points to partial-volume and resolution effects rather than contrast.

Baselines

The naive reference is the frozen 3D-MASNet source model applied directly to each target site. We also run it with per-subject histogram matching to the source, the fix iSeg-2019 participants used. iSeg-2019 gives the range of this regime, 0.82–0.86 multi-site test Dice against 0.90–0.92 validation. However, no reported result found on Lacuna for per-site Stanford-only Dice, so we re-measure it. Beating this reference requires a statistically significant gain in mean Dice on the GE site with no loss on Siemens sites.

The strongest existing method for few-label target adaptation is DuMeta (Dual Meta-Learning with Longitudinally Generalized Regularization for One-Shot Brain Tissue Segmentation Across the Human Lifespan). It meta-trains an age-agnostic encoder and a fast-adapting decoder across lifespan cohorts, then adapts the decoder with one labeled target scan. It reports one-shot iSeg-2019 Dice of 0.961 CSF, 0.931 GM and 0.914 WM, 2–5% above MAML and Reptile. It never isolates the cross-vendor split, which is exactly our setting. We run its one-shot decoder adaptation with one labeled GE scan, repeated over every choice of exemplar, and report the spread across exemplars as well as the mean. To be fair, it gets one labeled scan, while our method gets one unlabeled scan. If its meta-training cohorts are unavailable to us, we instead use a strong simpler surrogate: fine-tuning only the 3D-MASNet decoder on the one labeled scan, with early stopping tuned on Siemens validation data. Beating this baseline means matching its GE Dice within noise while having lower variance across exemplars and no source degradation. Exceeding its mean would be a bonus, not a requirement.

The strongest label-free competitor is masked pseudo-label self-training in the style of MAPSeg (3D Masked Autoencoding and Pseudo-labeling for Domain Adaptive Segmentation of Heterogeneous Infant Brain MRI). MAPSeg uses a teacher-student scheme with masked target inputs and reported a 10.5-point gain over prior domain adaptation, but it used 2,421 unlabeled scans. We restrict it to the same one to three target scans and the same GPU-hour budget as our fit, starting from the same source model.

Three ablations isolate the key ingredient. The first is a train-time counterpart: the source model is trained with random augmentations sampled from the same transform family $g_\phi$, and no test-time fitting is done. This asks whether fitting at test time matters, or whether the family is all that matters. The second ablation drops the volume and thickness terms in turn, leaving entropy only. The third restricts the family to per-channel curves without the coupling term, testing whether the joint (T1, T2) structure is necessary. All methods share the same source checkpoint and preprocessing, and each gets an equal hyperparameter search of 20 trials on Siemens validation subjects only. No method is tuned on GE data.

Why not just…?

Why not just fine-tune the network on one labeled target scan? This is the real threat, and we do not dismiss it. DuMeta shows that one labeled scan can carry a model to high iSeg-2019 Dice. Its known weakness is that a single exemplar's artifacts or atypical folding get absorbed into thousands of adapted weights. Fine-tuning can also erode source performance, which MAPSeg's authors report as forgetting during test-time adaptation. Our experiment measures both failure modes directly: Dice variance across the choice of exemplar, and change in Siemens Dice after adaptation. If one-shot fine-tuning wins on mean GE Dice with comparable variance, the honest conclusion is that the label is worth more than the prior. Our method would then keep its value only where labels are unavailable, and we would report it that way.

Why not just use stronger intensity augmentation during training? iSeg-2019 teams already tried tissue-dependent intensity augmentation, and it did not prevent the slump. Their augmentations were per-tissue scalings, though, not the joint nonlinear maps we hypothesize. If our hypothesis is right, augmenting over the right family might fix the problem with no test-time work at all. This is why the train-time counterpart ablation exists. If it matches test-time fitting, augmentation is the better recommendation and our contribution reduces to identifying the family. If it falls short, the per-site fit is earning its keep. One plausible reason it would fall short is that a near-isointense boundary cannot be made invariant to every sign flip at once.

Why not just run unsupervised domain adaptation like MAPSeg? MAPSeg's results rest on thousands of unlabeled target scans and heavy 3D masked-autoencoder pretraining, which is the opposite of a small infant site. With one to three target scans, self-training has little to average over, and pseudo-label drift is the expected failure. The matched-budget comparison measures this directly rather than assuming it.

Why not just use multi-atlas registration? Atlas methods do not need retraining, but they rely on image gradients that disappear in the isointense phase. The sparse multi-modal approach (Integration of Sparse Multi-modality Representation and Geometrical Constraint for Isointense Infant Brain Segmentation) found majority-vote atlas fusion inferior to its formulation, and that formulation also needed diffusion-derived fractional anisotropy maps, which clinical infant protocols often lack. We do not include atlases as a main baseline for that reason.

Proposed Experiments

The primary benchmark is iSeg-2019. We train on the 23 UNC/Siemens subjects and evaluate per site on the three test cohorts: UNC/UMN Siemens, Emory Siemens and Stanford GE. Metrics are Dice, modified Hausdorff distance and average surface distance per tissue, plus Dice restricted to straight and lingual gyrus regions. The test labels are held by the organizers, so this requires either their evaluation server or a data agreement, and securing that is the first milestone. If per-site test labels prove unavailable, the fallback is a cross-age, cross-site analogue built from the FeTA multi-center fetal data (Advances in automated fetal brain MRI segmentation and biometry: Insights from the FeTA 2024 challenge), splitting by acquisition site.

The first decisive check is the oracle. We fit $g_\phi$ on GE scans using their labels and evaluate on held-out GE scans. If the oracle closes less than half of the gap between GE and Siemens Dice, we stop pursuing label-free fitting and report that the slump is not mainly radiometric. That answer would also explain why intensity fixes failed in iSeg-2019.

The second check applies only if the oracle passes. We ask whether the label-free fit with $k=1$ reaches at least 70% of the oracle's gain, and whether the gain improves from $k=1$ to $k=3$.

The third check is whether the fitted method matches the one-labeled-scan DuMeta-style baseline within the confidence interval. It must do this with lower variance across exemplars and with Siemens-site Dice unchanged by construction, since the network itself is untouched.

The fourth check is the train-time augmentation counterpart, which decides whether fitting at test time is necessary at all.

We also report a calibration measure for each method on the GE site, because a fix that raises Dice while staying overconfident is clinically worse. Prior work shows out-of-domain segmentation calibration is poor: Improved post-hoc probability calibration for out-of-domain MRI segmentation reports expected calibration error of 14.48% under local temperature scaling under artifacts. Finally, we plot the fitted curves for each site. If Stanford's curves show a consistent, interpretable crossing of the T1 and T2 corrections that the Siemens sites lack, that is direct evidence for the mechanism we propose.