When a hospital joins a multi-site brain-imaging study, its scanner adds its own offset and scaling to every measurement. Before the hospital's data can be compared with anyone else's, these site effects have to be estimated and removed, a step called harmonization. Current tools estimate the site effects from whatever handful of local subjects the hospital has scanned, sometimes fewer than five. They then treat those estimates as exact. Every downstream number is reported as if the scanner's quirks were known perfectly. That includes a patient's percentile against a normative brain chart, an "abnormal" flag, and a forecast of cortical thinning. With three or four calibration subjects they are not known. Our claim is that this hidden estimation error dominates the error budget for exactly the patients clinicians care about, those far from the norm.
The fix we propose is to carry a posterior distribution over each new site's effects, rather than a point estimate, through to every downstream score. From that same distribution we derive a concrete stopping rule: the number of local subjects a site must scan before its scores are trustworthy at a stated tolerance. The work joins two literatures. One studies how errors in estimated parameters grow when they pass through nonlinear models. The other studies what can be learned from very small cohorts. Here the "parameter" is the scanner's site effect, and the "small sample" is the new site's calibration set.
Core Claim
Harmonization models of the ComBat family write a regional measurement, such as cortical thickness, for subject $i$ at site $s$ as $y_{is} = \alpha + x_i^\top\beta + \gamma_s + \delta_s\,\varepsilon_{is}$. Here $\alpha$ is the global mean, $x_i$ are biological covariates with effects $\beta$, $\gamma_s$ is the site's additive shift, $\delta_s$ is its multiplicative scale, and $\varepsilon_{is}$ is noise. A patient's standardized score is $z = (y - \hat\alpha - x^\top\hat\beta - \hat\gamma_s)/(\hat\delta_s\hat\sigma)$, where $\hat\sigma$ is the pooled residual spread.
Suppose $\gamma_s$ and $\delta_s$ are estimated from $n$ local subjects. The delta method for a Gaussian location and scale then gives the extra variance in the patient's score:
$$ \mathrm{Var}(\hat z \mid z) \approx \frac{1}{n}\left(1 + \frac{z^2}{2}\right). $$
The $1/n$ comes from estimating the mean. The $z^2/2n$ comes from estimating the spread, whose error is multiplied by how far the patient sits from the centre. For $n = 3$ and a true score of $z = -2$, the standard deviation of the computed score is about 1. That is large enough to move a patient across a clinical threshold by chance alone. Two things soften this in practice. Shrinkage toward a prior learned from many existing sites reduces the error, and the prior can itself be wrong. Our first claim is that, even after this shrinkage, the error is large and concentrated in the tails at the site sizes that current methods advertise as workable.
Our second claim is that integrating over the posterior of $(\gamma_s, \delta_s)$, rather than plugging in its mean, yields calibrated intervals for each site separately. This has to hold at every site, not only on average across sites. The same posterior also gives a per-site, per-region estimate of how many more subjects would bring the score error below a chosen tolerance.
Relation to Prior Work
The closest work is ComBat-Predict enhances generalizability of neuroimaging models to new sites. It supplies the method we build on. It keeps the global and site parameters learned from the original sites fixed, then estimates a new site's adjustment by method of moments without re-pooling the training data. Across 505 ADNI subjects from 64 sites, it reports good harmonization with only 3 to 20 subjects per test site and better centile alignment against the Lifespan Brain Chart Consortium (LBCC) normative model of 12,396 adults. It also motivates our proposal, because it outputs deterministic harmonized values and never asks how wrong a three-subject site estimate is. We keep its decoupled structure and replace its point estimate with a posterior. We also add something it cannot provide: a sample-size recommendation for each site.
Several papers show that small parameter errors become large output errors once they pass through nonlinear maps. Robust Lock-Down Optimization for COVID-19 Policy Guidance shows that a 5% error in a transmission rate flips projected epidemic outcomes. Exact vs. Approximate Computation shows that different estimators applied to the same tuberculosis data give conflicting reproductive numbers. Both are motivation by analogy. Thresholded clinical scores and learned trajectory models are nonlinear in the same sense, so our object of study is error in an estimated nuisance parameter rather than in a biological one.
Conditional Score-Based Diffusion Model for Cortical Thickness Trajectory Prediction supplies our nonlinear downstream model. It is a generative model that forecasts thickness across 68 brain regions on the TADPOLE cohort, which is derived from ADNI, and it expresses uncertainty through repeated stochastic samples. Its uncertainty describes biological variation only, and its inputs are assumed already harmonized. We test whether adding site-parameter uncertainty changes its forecast intervals in a clinically meaningful way. This bears on a puzzle it reports: it trails a deterministic model on the Mild Cognitive Impairment (MCI) subgroup.
The fMRI-decoding work on small calibration sets, MindAligner and MindEye2, shows that one hour of target-subject data is enough when strong cross-subject priors exist. This is evidence that priors pooled across many units make tiny calibration sets usable. Neither paper quantifies how uncertain the resulting alignment is, and that quantity is what we measure.
Research Program
We will build a hierarchical empirical-Bayes model of site effects. It will be fitted once on the training sites and will give a prior over $(\gamma_s, \delta_s)$ for each brain region, with correlations across regions so that information is shared between neighbouring regions. When a new site supplies $n$ subjects, we compute the posterior in closed form under conjugate assumptions. As a check on those assumptions, we also compute it by sampling, with a heavier-tailed prior that is robust to a mis-specified site population.
One subtlety matters a great deal at tiny $n$. The patient being scored is usually one of the $n$ subjects used to estimate the site effects, so the patient's own value pulls the site mean toward itself and shrinks its apparent abnormality. For every scored subject we will compute the site posterior with that subject left out. We will quantify how much this self-influence understates abnormality.
Downstream, we draw site-parameter samples, harmonize under each one, and recompute three quantities: centile scores against an LBCC-style normative model, the binary flag "below the 2.5th centile", and trajectory forecasts from a reimplementation of the conditional diffusion model. From the posterior variance of these outputs we derive a stopping rule: the smallest $n$ at which the expected variance of a region's score at a reference abnormal level, say $z = -2$, falls below a chosen tolerance. We will check this rule against held-out data.
Baselines
The naive reference point applies no harmonization. The new site's raw measurements go directly into the normative and trajectory models. ComBat-Predict already shows that this inflates prediction error relative to harmonized data. We include it to scale the size of every other effect.
The strongest existing method is ComBat-Predict itself, run as published with method-of-moments site estimates on the same leave-one-site-out splits and the same subsampled site sizes. It is the strongest competitor because it is the only method in the file designed for new sites with fewer than five subjects. We extend it with the most favourable uncertainty heuristic we can construct, a nonparametric bootstrap over the site's subjects. Our goal is to beat it on calibration, not on accuracy. At matched mean error, our intervals should reach their nominal coverage at each site separately, and our abnormality flags should have fewer threshold crossings caused by site estimation alone. The file contains no numeric RMSE for ComBat-Predict; it reports only "significantly lower RMSEs compared to using unharmonized data" and centile scores "more centered around the expected." For that reason we will reproduce its numbers ourselves rather than quote them.
For the trajectory experiments, the reference is the conditional diffusion model on harmonized inputs, which reports a mean absolute error of 0.092 across 68 regions against 0.117 for GRUD and 0.120 for LSTM-T (Conditional Score-Based Diffusion Model). We will retrain all variants of the model with the same budget. The aim is to keep its accuracy while achieving correct interval coverage at sites with $n \le 5$.
Two ablations isolate what is new. The first uses the posterior mean of the site effects as a plug-in point estimate. This keeps the shrinkage and drops the propagation, and tells us whether shrinkage alone accounts for any gains. The second turns off the leave-self-out correction, which isolates the self-influence effect.
Why not just…?
Why not just bootstrap ComBat-Predict over the site's subjects? With three subjects, a bootstrap sees at most a few distinct resamples. It cannot borrow strength from the other 63 sites, so its scale estimate is erratic and its intervals are badly calibrated. We expect the bootstrap to under-cover at $n = 3$ and approach the posterior method by about $n = 15$. We will measure where the curves meet rather than assume it. If the bootstrap matches our method at $n = 5$, the contribution shrinks to the stopping rule and the self-influence finding.
Why not just apply conformal prediction to the downstream residuals? Leave-one-site-out split conformal is a real threat. It guarantees coverage averaged over sites without any model of the site. It gives the same interval width to a three-subject site and a twenty-subject site, so it should over-cover large sites and under-cover small ones. This is exactly the failure that matters. We will report coverage for each site, plotted against $n$, for both methods. If conformal intervals turn out to be well calibrated per site, we will adopt them and keep only the stopping-rule analysis.
Why not just pool all sites and let a deep model learn site-invariant features? ComBat-Predict already shows that unharmonized pooling distorts centile scores for healthy controls at new sites. More fundamentally, an implicitly invariant model still has to infer an unseen scanner's effect from that scanner's few subjects. The estimation error remains, only hidden inside the model. We include pooling as the naive baseline.
Proposed Experiments
All experiments use ADNI cortical thickness with ComBat-Predict's protocol. We hold out one site at a time and subsample its subjects to $n \in \{3, 5, 10, 20\}$, with many random draws per size. The site's full sample serves as a high-$n$ reference for its "true" site effects, which is possible only for held-out sites that are large. Normative centiles use an LBCC-style chart. Trajectory experiments use TADPOLE's 720/178 split, with site labels kept and subsampled in the same way.
The metrics are as follows. We measure per-site interval coverage and width for centiles. We count false abnormal flags among healthy controls, and missed flags among AD patients, attributable to site estimation. We measure trajectory MAE and coverage, reported separately for MCI. Finally, we measure the prediction error of the stopping rule: does the recommended $n$ actually achieve the tolerance on held-out draws?
The first decisive check tests the core claim. With ComBat-Predict at $n \le 5$, the variance in $z$ due to the site should grow with $|z|$ roughly as the formula predicts, and the fraction of flips across the 2.5th-centile threshold should be substantial. If site-induced variance is small compared with biological variance even at $n = 3$, the problem is not worth solving and we will say so.
The second check asks whether propagated intervals keep their nominal coverage at each site while plug-in and conformal intervals do not.
The third check asks whether adding site uncertainty to the diffusion model widens MCI intervals enough to change which patients are labelled fast decliners. A positive result would mean that part of what that paper attributes to biological instability in MCI is scanner uncertainty instead.
References
- ComBat-Predict enhances generalizability of neuroimaging models to new sites
- Robust Lock-Down Optimization for COVID-19 Policy Guidance
- Exact vs. Approximate Computation: Reconciling Different Estimates of Mycobacterium tuberculosis Epidemiological Parameters
- Conditional Score-Based Diffusion Model for Cortical Thickness Trajectory Prediction
- MindAligner: Explicit Brain Functional Alignment for Cross-Subject Visual Decoding from Limited fMRI Data
- MindEye2: Shared-Subject Models Enable fMRI-To-Image With 1 Hour of Data