ComBat-Predict enhances generalizability of neuroimaging models to new sites

Y. Xin, Margaret Gardner, Nicholas J. Tustison, Philip A. Cook, James C. Gee, Andreana Benitez, Jens H. Jensen, Richard A. I. Bethlehem, Jakob Seidlitz, Aaron Alexander‐Bloch, Andrew A. ChenbioRxiv 2025 · DOI

TL;DR

This paper introduces ComBat-Predict (CB-Predict), a novel method for harmonizing neuroimaging data from new, unseen sites without needing access to the original data. CB-Predict effectively reduces site-specific biases, improving the generalizability of predictive models and normative analyses to new datasets, even with small sample sizes.

What Problem Does This Paper Address?

Neuroimaging studies often aggregate data from multiple sites to increase sample sizes and improve statistical power. However, this multi-site design introduces "site effects" – systematic, non-biological variations due to differences in scanners, acquisition parameters, or processing pipelines. These effects can mask true biological differences and lead to biased results. While existing methods like ComBat can harmonize data from known sites, they struggle to generalize to new sites with unknown characteristics or small sample sizes, limiting their real-world applicability. This paper addresses the critical need for a harmonization method that can adapt to novel sites without requiring their data to be pooled with the original training set.

Methods

The core of this work is the ComBat-Predict (CB-Predict) method, an extension of the established ComBat framework. CB-Predict builds upon the empirical Bayes approach of ComBat but is designed to estimate harmonization parameters for a new site without direct access to its data.

Here's a breakdown:

  • Standard ComBat: The original ComBat method uses an empirical Bayes approach to adjust for additive and multiplicative site effects. It models the data as having an overall mean, site-specific additive effects, and site-specific multiplicative effects, along with residual error.
  • CB-Predict Extension: CB-Predict leverages the harmonization parameters (mean, additive effects, multiplicative effects) learned from existing "original" sites. When a new site is encountered, CB-Predict uses these learned parameters to standardize the new site's data. It then estimates the site-specific adjustment parameters for this new site using a method of moments approach on the standardized data. Crucially, this estimation is done independently of the original data.
  • Generalization: The standardized data from the new site can then be harmonized using these newly estimated parameters. The paper also notes that CB-Predict can be extended to incorporate more complex models, such as Generalized Additive Models (GAMs) for non-linear relationships, and can also work with subject-specific standard deviation estimates (as in ComBatLS).

Datasets

The study utilizes two major neuroimaging datasets:

  • Alzheimer's Disease Neuroimaging Initiative (ADNI): This dataset was used for evaluating CB-Predict's ability to harmonize data from novel sites and improve the generalizability of predictive models. The ADNI dataset, comprising T1-weighted MRI scans from participants with Alzheimer's disease and related conditions, was processed to extract cortical thickness measures for 62 brain regions. For this study, 505 subjects from 64 sites were used after filtering for sample size.
  • Lifespan Brain Chart Consortium (LBCC): This large-scale dataset, containing over 1.2 million structural MRI scans from over 100,000 participants, was used to establish a normative model. A subset of 12,396 healthy adults from 45 studies was created for this analysis. The LBCC data was used to train a normative model that was then applied to the ADNI data.

Key Insights and Ideas

  • Harmonization Without Data Pooling: The most significant insight is that effective site effect harmonization can be achieved for new sites without needing to pool their data with existing datasets. This is a major step forward for data sharing and collaborative research, especially when data privacy or logistical constraints are an issue.
  • Robustness to Small Sample Sizes: CB-Predict demonstrates impressive performance even with very small sample sizes (as low as n < 5) in the new site, a common scenario in real-world data collection.
  • Improved Predictive Model Generalizability: By harmonizing data from a new site, CB-Predict significantly improves the accuracy of predictive models trained on harmonized data from other sites. This means models developed in one setting are more likely to perform well in a new, unseen setting.
  • Better Normative Model Alignment: CB-Predict helps align centile scores derived from a new site's data with those from a larger reference population. This is crucial for normative modeling, ensuring that an individual's brain metrics are compared against a consistent and representative baseline, regardless of where the data was collected.
  • Reduced Site Heterogeneity: The method effectively reduces site-related heterogeneity in centile scores, making comparisons between healthy control groups from different cohorts more reliable.

Key Results

  • Low RMSE in Harmonization: In a leave-one-site-out cross-validation on ADNI data, CB-Predict consistently achieved low Root Mean Squared Errors (RMSE) when comparing its harmonization to standard ComBat applied to all sites. This held true across sites with varying sample sizes (3-20 subjects) and diagnostic compositions.
  • Lower Prediction Errors: When predicting cortical thickness measures using a multivariate linear model trained on harmonized ADNI data, CB-Predict harmonization of the test site data resulted in significantly lower RMSEs compared to using unharmonized data.
  • Improved Centile Score Alignment: In normative modeling, harmonizing ADNI data with CB-Predict led to centile scores for healthy controls that were more centered around the expected 0.5 and showed reduced variance, aligning better with the reference LBCC control group.
  • Reduced Heterogeneity in Healthy Controls: Quantitative analysis using rank biserial correlation showed that CB-Predict harmonization substantially reduced the heterogeneity between healthy control groups from the LBCC and ADNI cohorts, indicating greater similarity in their normative centile score distributions.

Limitations and Future Work

  • ADNI-Centric Analysis: The primary analyses were conducted on the ADNI dataset, which primarily represents populations from the United States. The authors suggest future work should evaluate CB-Predict's generalizability on datasets with more diverse international populations, such as the ENIGMA study.
  • Extension to Other ComBat Variants: While CB-Predict is designed to work with other ComBat extensions (like ComBat-GAM and ComBatLS), its performance with these specific variants was not directly evaluated in this study. Future research could explore this.
  • Single Regression Approach for Centiles: The study used a single regression-based approach for generating centile scores. Future work could investigate the impact of CB-Predict harmonization on the robustness and interpretability of centile scores derived from alternative normalization methods (e.g., quantile regression, hierarchical modeling, variational autoencoders).

Summary updated

Abstract

Neuroimaging is vital in quantifying brain atrophy due to typical aging and due to neurodegenerative diseases. To collect large samples necessary to model lifespan brain development, research consortiums aggregate images acquired across multiple study sites. Previous studies have demonstrated that this multi-site study design can lead to site-related bias, necessitating harmonization of these "site effects". However, current methodologies are unable to generalize to new sites outside the original harmonized sample, limiting translation to new sites or clinical practice. Here, we propose a method called ComBat-Predict (CB-Predict) building upon the ComBat method for site effect adjustment, which extends to data from a new site with smaller sample sizes and unknown site effects. In data from the Alzheimer's Disease Neuroimaging Initiative (ADNI), our proposed method mitigates bias and yields high accuracy in predicting cortical thickness measures when generalizing the model to new data. Furthermore, we demonstrate that our proposed harmonization method can reduce site-related variance in centile scores estimated using data from the Lifespan Brain Chart Consortium (LBCC). Altogether, our results demonstrate that CB-Predict effectively harmonizes new sites and thereby enables effective translation of neuroimaging models to additional samples.
BibTeX
@misc{xin2025combat,
  author = {Y. Xin and Margaret Gardner and Nicholas J. Tustison and Philip A. Cook and James C. Gee and Andreana Benitez and Jens H. Jensen and Richard A. I. Bethlehem and Jakob Seidlitz and Aaron Alexander‐Bloch and Andrew A. Chen and Andrew An Chen},
  title = {{ComBat-Predict enhances generalizability of neuroimaging models to new sites}},
  year = {2025},
  doi = {10.1101/2025.08.21.671401},
  howpublished = {bioRxiv},
}
Figure