Increased power by harmonizing structural MRI site differences with the ComBat batch adjustment method in ENIGMA

Vince D. Calhoun, Joaquim Radua, Eduard Vieta, Russell T. Shinohara, Peter V. Kochunov, Yann Quidé, Melissa J. Green, Cynthia S. Weickert, Thomas Weickert, Jason Bruggemann, Tilo Kircher, Igor Nenadic, Murray J. Cairns, Marc L. Seal, Ulrich Schall, Frans Henskens, Janice M. Fullerton, Bryan J. Mowry, Christos Pantelis, Julian Pineda-ZapataNeuroImage 2020 · DOI

TL;DR

This paper introduces a novel application of the ComBat batch adjustment method to harmonize structural MRI data across multiple research sites within the ENIGMA consortium. By effectively reducing site-specific biases, ComBat significantly increases statistical power and the reliability of findings in large-scale neuroimaging studies, particularly for detecting subtle differences in brain structure associated with conditions like schizophrenia.

What Problem Does This Paper Address?

Neuroimaging studies, especially those investigating complex conditions like schizophrenia, often suffer from small sample sizes, limiting their statistical power to detect true differences and increasing the risk of false positives. To overcome this, large consortia like ENIGMA pool data from numerous institutions worldwide. However, this pooling introduces significant heterogeneity due to variations in MRI scanners, acquisition protocols, and processing pipelines across sites. While methods like random-effects meta-analysis (RE-Meta) and mixed-effects mega-analysis (ME-Mega) are used to address this, they may not fully account for all sources of site-related variability. This study aims to investigate whether ComBat, a statistical method originally developed for genomics, can further reduce this heterogeneity and thus enhance statistical power in structural MRI mega-analyses.

Methods

The researchers compared three analytical approaches for detecting differences in brain structure between individuals with schizophrenia (SCZ) and healthy controls (CON) across 33 sites:

  • Random-Effects Meta-Analysis (RE-Meta): This method pools effect sizes (Hedges' g) from individual site analyses. It accounts for between-site heterogeneity but can be suboptimal for site-aggregated data.
  • Mixed-Effects Mega-Analysis (ME-Mega): This approach uses linear mixed-effects models on individual subject data, treating "site" as a random factor. It offers more robust estimation but assumes similar error distributions across sites.
  • ComBat Mega-Analysis (ComBat-Mega): This is the novel approach proposed. It applies the ComBat algorithm to harmonize the data before standard linear modeling. ComBat adjusts for site effects by modeling both additive and multiplicative batch effects, assuming that these effects share a common distribution across regions. The authors also developed modified ComBat functions to handle missing data, which is common in neuroimaging datasets.

The study also employed a permutation test to assess statistical significance and a small-subset strategy to evaluate statistical power.

Datasets

The study utilized individual subject-level structural MRI data from the ENIGMA Schizophrenia Working Group. This dataset comprised:

  • 2897 individuals with a diagnosis of schizophrenia (SCZ)
  • 3141 healthy controls (CON)

Data included cortical thickness, surface area, and subcortical volumes from 33 participating sites. The data was processed using FreeSurfer pipelines according to ENIGMA protocols.

Key Insights and Ideas

  • ComBat's Power Boost: The most significant insight is that ComBat harmonization substantially increases statistical significance and power compared to traditional RE-Meta and, to a lesser extent, ME-Mega. This suggests ComBat is more effective at removing site-specific biases.
  • Site Effects are Complex: The study highlights that even with standardized preprocessing, systematic differences between MRI scanners and acquisition sequences create significant heterogeneity. ComBat's ability to model both additive and multiplicative effects seems crucial for addressing this.
  • Impact of Intra-Site Variance: The increase in statistical significance with ComBat was particularly pronounced in regions where intra-site variance constituted a smaller proportion of the total variance (around 50-70%). This implies ComBat is most beneficial when site effects are a substantial contributor to overall variability.
  • Improved Power with Fewer Sites: ComBat-Mega demonstrated increased statistical power when analyses were repeated with a reduced number of sites, indicating its robustness and effectiveness even with limited data.
  • Handling Missing Data: The development of modified ComBat functions to accommodate missing data is a critical practical advancement for neuroimaging applications.

Key Results

  • ComBat-Mega identified widespread thinner cortex and smaller surface area in individuals with SCZ across nearly all cortical regions, consistent with previous ENIGMA findings but with greater statistical significance.
  • The statistical significance of findings was substantially greater in ComBat-Mega compared to RE-Meta (median difference in logit-transformed p-values of 13.9) and moderately greater compared to ME-Mega (median difference of 3.2).
  • Statistical power was significantly higher for ComBat-Mega (83.5%) compared to RE-Meta (53.7%) and ME-Mega (80.4%) when evaluated using a small-subset strategy.
  • All methods controlled the familywise error rate (FWER) appropriately, likely due to the conservative Holm correction.

Limitations and Future Work

  • Exploration of Other Data Types: The study focused on cortical thickness, surface area, and subcortical volumes. The authors acknowledge that the benefits of ComBat for other neuroimaging data types (e.g., diffusion MRI, functional connectivity) are not yet explored, though evidence from other fields suggests broad applicability.
  • Single Harmonization for Mixed Data: While a single ComBat harmonization was applied across different MRI data types (thickness, area, volume), the authors suggest caution when combining other data modalities, as the behavior of a single harmonization might differ.
  • Site by Nuisance Confounds: ComBat addresses site effects but not necessarily other site-specific confounds (e.g., site-specific age distributions or data quality issues). Future research could explore methods to address these.
  • Data Availability for Voxel-Based Methods: The authors note that while voxel-based meta-analytic methods (like SDM and ALE) can include more studies, they typically require only summary statistics or peak coordinates. ComBat-Mega, however, requires individual subject data, which is not always available. This presents a challenge for applying ComBat to the full spectrum of published neuroimaging literature.

The authors recommend that ENIGMA projects and other multi-site neuroimaging studies consider adopting ComBat harmonization to enhance statistical power and the reliability of their findings. They also provide R code for their adapted ComBat functions to facilitate its use.

Summary updated

BibTeX
@article{calhoun2020increased,
  author = {Vince D. Calhoun and Joaquim Radua and Eduard Vieta and Russell T. Shinohara and Peter V. Kochunov and Yann Quidé and Melissa J. Green and Cynthia S. Weickert and Thomas Weickert and Jason Bruggemann and Tilo Kircher and Igor Nenadic and Murray J. Cairns and Marc L. Seal and Ulrich Schall and Frans Henskens and Janice M. Fullerton and Bryan J. Mowry and Christos Pantelis and Julian Pineda-Zapata},
  title = {{Increased power by harmonizing structural MRI site differences with the ComBat batch adjustment method in ENIGMA}},
  journal = {NeuroImage},
  year = {2020},
  doi = {10.1016/j.neuroimage.2020.116956},
}
Figure