Brain models that combine anatomy and activity often use a single number per brain region: how strongly the region's wiring, measured by structural or diffusion MRI, predicts its activity patterns, measured by functional MRI. When scans come from many hospitals with different scanners, that number shifts from site to site. Part of the shift reflects real differences in the patients. Part reflects the hardware. Current multimodal models learn one coupling value per region and assume every site shares it.
We propose treating each site's regional coupling the way a meta-analyst treats a study's effect size. Each site gives an estimate with a known sampling error. A random-effects model, which lets the true value differ across sites, then measures how much it really differs. These estimates are used in two ways. They map which regional couplings depend on the scanner. They also set a continuous shrinkage rule inside the network. That rule decides how far a new site's coupling may move away from the global value, and it can be computed from unlabeled scans at that site.
Core Claim
Meta-analytic tools for heterogeneity need scalar summary statistics with known variances. That requirement is usually why people say these tools "do not extend to neuroimaging tensors or graphs." Regional structure–function coupling meets the requirement exactly. For region $r$ at site $s$, let $y_{s,r}$ be the across-subject mean correlation between that region's structural connectivity profile and its functional connectivity profile, after regressing out age and sex. Let $V_{s,r}$ be its squared standard error. We fit, region by region,
$$ y_{s,r} = \mu_r + x_s^\top \beta_r + u_{s,r} + \varepsilon_{s,r}, \qquad u_{s,r} \sim N(0,\tau_r^2),\ \varepsilon_{s,r} \sim N(0, V_{s,r}), $$
where $\mu_r$ is the global coupling, $x_s$ holds site-level moderators, and $\tau_r^2$ is the leftover between-site variance. The moderators are biological (mean age, sex ratio, diagnosis proportion, diagnostic instrument) and technical (vendor, field strength, sequence). Fitting $\beta_r$ splits the heterogeneity into a biological part and a technical part, which is the decomposition adversarial harmonization cannot provide.
Inside a two-layer graph network, the coupling used for site $s$ is
$$ c_{s,r} = c_r + \lambda_r\, w_{s,r}\, d_{s,r}, \qquad w_{s,r} = \frac{\tau_r^2}{\tau_r^2 + V_{s,r}}. $$
Here $c_r$ is the learned global coupling, $d_{s,r}$ is the site's deviation $y_{s,r} - \mu_r - x_s^\top\beta_r$ explained only by technical moderators, and $\lambda_r$ is a learned scale. The weight $w_{s,r}$ is the standard empirical-Bayes shrinkage factor. It is the continuous version of the "switch to random effects when $I^2 \ge 50\%$" rule used across clinical meta-analyses such as Leonard Chiu et al. and Jiawei Qi et al.. It removes the threshold jump where a small change in $I^2$ flips the weights. Because $d_{s,r}$ needs only scans and not diagnoses, the model can adapt to an unseen site from unlabeled data.
The bet is that site effects in multimodal models enter mainly through fusion: scanners distort diffusion and functional MRI differently, so the relative trust between the two layers is what drifts. If so, putting random effects on a few dozen coupling parameters should remove most of the shortcut signal while leaving the diagnostic signal intact.
Positioning
The architecture comes from Joint-GCN. It builds a 2N-node graph with a structural layer and a functional layer, joined by learnable per-region coupling edges. On NCANDA it reached 84.9% sex-classification accuracy and an age-prediction mean absolute error of 1.95 years. Its coupling maps were biologically sensible, but it pooled all scans under one coupling vector. We keep its message passing and make the coupling vary by site, with a hierarchical prior.
The random-effects-in-deep-networks idea comes from LMMNN. LMMNN adds a group random effect $g(Z)b$ to a network's output and fits it by Gaussian likelihood, beating one-hot site codes and entity embeddings on tabular data. We put the random effect on an internal, interpretable fusion parameter instead of the output, and we test that choice directly.
Three papers motivate the work. Batch Effects In Brain Foundation Model Embeddings found that functional MRI foundation-model embeddings predict site with 85–94% accuracy. On FBIRN, site Pseudo-F was 717.35 against 0.15 for diagnosis, and post-hoc ComBat on embeddings did not recover biology. Detect and Correct Bias in Multi-Site Neuroimaging Datasets found 71.5% source-dataset decodability across 17 cohorts. Brain ageing in schizophrenia via ENIGMA found $I^2 = 57.53\%$ when pooling a model-derived brain-age score across 26 cohorts. That ENIGMA study is the closest precedent for meta-analyzing a learned quantity across sites, but it does so after training, on one structural MRI score. We do it inside the multimodal model, on the fusion parameters.
Is UWLS Really Better for Medical Research? constrains how we estimate. It found that information criteria pick the true random-effects model only 8.7% (AIC) and 9.4% (BIC) of the time with about five studies, and that random-effects intervals cover about 93.4% against 85.6% for UWLS-FE. We therefore never select fixed versus random by AIC or BIC. We always fit random effects and let $w_{s,r}$ shrink toward zero when $\tau_r^2$ is small.
Because ADHD-200 has only 7–8 sites, we estimate $\tau_r^2$ by restricted maximum likelihood rather than DerSimonian–Laird, which is unreliable with few studies. We also pool information across regions with a weak hyperprior on $\log\tau_r^2$.
Research Program
Stage one: the heterogeneity map. Using code from DataCentricBrainGraphs, we build functional graphs with rank correlations, which Ge et al. found work better than Pearson. Structural layers come from diffusion tractography where it exists. NCANDA provides diffusion MRI and resting-state functional MRI and is collected at several acquisition sites, though Joint-GCN did not model them. For ABIDE and ADHD-200, which the file documents as having T1 structural MRI and functional MRI, we use morphometric-similarity graphs from T1 scans.
For each region we report the global coupling $\mu_r$, $\tau_r^2$, $I^2_r$, and how much heterogeneity the biological versus technical moderators explain. The scientific output is a cortical map of which coupling estimates can and cannot be compared across scanners. Its validity is checked against the biological pattern reported by Joint-GCN: high coupling in superior parietal and frontal cortex, low in olfactory cortex.
Stage two: site-aware coupling in the network. We train the joint graph network with the rule above. $\tau_r^2$ is re-estimated each epoch from the training sites, so the shrinkage tracks the data rather than a fixed preprocessing step.
Stage three: adapting to an unseen site. At a held-out site, we compute $d_{s,r}$ from its unlabeled scans, apply the shrinkage, and run inference. We do not touch labels or retrain weights.
Baselines
The naive reference is Joint-GCN trained on pooled data with no site handling. On NCANDA the comparable reported numbers are 84.9% sex accuracy and AUC 0.907 (Joint-GCN). It shows how much shortcut learning the plain architecture does under leave-one-site-out testing.
The strongest simple competitor is ComBat-GAM applied as offline preprocessing to every structural and functional edge, with age modeled nonlinearly, followed by the same Joint-GCN. Sun et al. showed ComBat-GAM finds 25 PTSD-affected regions against 6 for linear ComBat. It also recovered Age×Diagnosis effects that linear methods missed entirely. Its weaknesses are the ones our method targets. It corrects each modality separately, so it cannot see miscalibration between the layers. It needs every site's data at once. And it risks erasing signal when diagnosis is unbalanced across sites.
The strongest published classifier is SI-IGCL, which reports 75.27% accuracy on ABIDE (1,009 subjects) and 74.26% on ADHD-200 (685 subjects), both with the Craddock 200 atlas. We rerun it from its description under our leave-one-site-out splits and parcellation rather than copying its numbers, because its reported accuracy is not tied to a single protocol we can reproduce exactly. The older multimodal kernel method from Dai et al. (67.79%, AUC 0.7068 on ADHD-200) is a weak floor. We also include a gradient-reversal adversarial site remover on the joint network as the standard learned-invariance competitor.
Three ablations isolate the key ingredient. The first is an LMMNN-style random intercept per site on the network output, which tests whether the random effect belongs on the coupling or the output. The second is a fixed, unshrunk per-site coupling, which tests whether shrinkage matters. The third is a hard switch that uses site-specific coupling only where $I^2_r \ge 50\%$, which tests whether continuous weighting beats the conventional threshold.
All methods share the same splits, parcellation, graph construction, and a 50-configuration hyperparameter search per method on inner folds. To beat a baseline, our method must improve mean leave-one-site-out balanced accuracy, with the confidence interval over held-out sites excluding zero. It must do this without raising embedding site-decodability above the ComBat-GAM pipeline.
Why not just…?
Why not just run ComBat-GAM on the connectome edges and then train Joint-GCN? This is the real threat. ComBat works well on raw tabular features (Sun et al., Wachinger et al.); its documented failure in Batch Effects In Brain Foundation Model Embeddings is on deep embeddings. That failure does not show it fails on edges. Our case rests on three points ComBat-GAM cannot cover: label-free adaptation to an unseen site without refitting on everyone, avoiding over-correction when diagnosis is nearly confounded with site, and an interpretable split of biological versus technical heterogeneity. If ComBat-GAM ties us on balanced accuracy and confounded-split robustness, we will report that the heterogeneity map is the contribution and the network component is not needed.
Why not just add an LMMNN random site intercept to the output? LMMNN shows output-level random effects beat one-hot and embedding site codes. But an output intercept only shifts predictions. It cannot stop the encoder from using scanner-specific fusion as a diagnostic shortcut, which the confounded-site test exposes directly. This alternative is our first ablation, so the comparison is settled within the experiments.
Why not just use adversarial removal or SI-IGCL's inverse contrastive learning? Both reduce site information empirically. Neither reports how much heterogeneity exists, where it sits, or how much of it is biological. SI-IGCL's objective, which pushes apart embeddings of the same subject, may also erase true clinical subtypes. If either beats us on accuracy, our method still gives the variance decomposition. We would then test whether adding our coupling shrinkage on top of SI-IGCL helps, rather than claiming to replace it.
Decisive Checks
The first check is Tao et al.'s confounded split, where all patients come from one site and all controls from another. A model that learns scanners scores near 100% in-split and collapses on a balanced held-out site. Ours should degrade least. The second is a linear probe for site on the fused embeddings. The third is a meta-analysis of each model's per-site diagnostic effect size, its Cohen's $d$ between patients and controls. A good model should lower the technical part of $I^2$ while keeping the pooled effect.
The decisive falsification test is simulation. We inject known scanner distortions with chosen $\tau_r^2$ into a subset of regions on NCANDA, then measure whether the map recovers those regions, whether the moderator split attributes them to scanner, and whether the predicted between-site intervals reach about 95% coverage. With only 7–8 sites in ADHD-200, we expect wide $\tau_r^2$ intervals. If coverage on ADHD-200 falls well below nominal even with restricted maximum likelihood and cross-region pooling, the method is limited to consortia with many sites, such as ABIDE. We would state that limit plainly.
References
- Colchicine use in patients with COVID-19: A systematic review and meta-analysis
- Selection of timing of continuous renal replacement therapy in patients with acute kidney injury: A meta-analysis of randomized controlled trials
- Joint Graph Convolution for Analyzing Brain Structural and Functional Connectome
- Integrating Random Effects in Deep Neural Networks
- Batch Effects In Brain Foundation Model Embeddings
- Detect and correct bias in multi-site neuroimaging datasets
- Brain ageing in schizophrenia: evidence from 26 international cohorts via the ENIGMA Schizophrenia consortium
- Is <scp>UWLS</scp> Really Better for Medical Research?
- Defining and Benchmarking a Data-Centric Design Space for Brain Graph Construction
- A comparison of methods to harmonize cortical thickness measurements across scanners and sites
- SI-IGCL: Subject Invariance-aware Inverse Graph Contrastive Learning for Psychiatric Disorder Identification
- Classification of ADHD children through multimodal magnetic resonance imaging