DeepComBat: A Statistically Motivated, Hyperparameter-Robust, Deep Learning Approach to Harmonization of Neuroimaging Data

Fengling Hu, Alfredo Lucas, Andrew A. Chen, Kyle Coleman, Hannah Horng, Raymond W.s. Ng, Nicholas J. Tustison, Kathryn A. Davis, Haochang Shou, Mingyao Li, Russell T. Shinohara, The Alzheimer’s Disease Neuroimaging InitiativebioRxiv 2023 · DOI

TL;DR

This paper introduces DeepComBat, a novel deep learning approach to harmonize neuroimaging data collected across different scanners or sites. By combining deep learning with established statistical methods, DeepComBat effectively removes technical "batch effects" while preserving crucial biological information, outperforming existing methods in key analyses.

What Problem Does This Paper Address?

Neuroimaging studies increasingly rely on data from multiple sources (e.g., different hospitals, scanner manufacturers) to achieve larger sample sizes and improve generalizability. However, these "multi-batch" datasets are plagued by technical artifacts, known as "batch effects," which can confound results and obscure true biological signals. Existing harmonization methods, both statistical and deep learning-based, often struggle to fully remove these effects, especially in complex, multivariate analyses like those used in machine learning. This can lead to unreliable or irreproducible findings.

Methods

DeepComBat is a hybrid approach that leverages a Conditional Variational Autoencoder (CVAE) architecture, inspired by deep learning, and integrates it with the well-established ComBat statistical harmonization model.

Here's a breakdown:

  • CVAE for Latent Space Learning: The CVAE learns a compressed, "latent" representation of the neuroimaging data. This latent space is designed to capture subject-specific biological information while minimizing batch-related artifacts. The CVAE also learns to reconstruct the original data from this latent representation, using both biological and batch information.
  • ComBat for Harmonization: Crucially, DeepComBat applies ComBat not only to the raw data but also to the learned latent space and the reconstruction residuals (the parts of the data not perfectly reconstructed by the CVAE). This multi-stage harmonization allows for a more robust removal of batch effects.
  • Relaxed Assumptions: Unlike some deep learning methods that assume perfect model fit or complete disentanglement of batch and biological information, DeepComBat is designed to be more forgiving. It explicitly accounts for reconstruction residuals and doesn't require a perfectly batch-invariant latent space, making it more practical and robust.

Datasets

The primary dataset used in this study is from the Alzheimer's Disease Neuroimaging Initiative (ADNI). This dataset consists of T1-weighted MRI scans from 663 subjects. The study specifically defines two batches based on scanner manufacturer: Siemens Healthineers and a combined batch of Philips Medical Systems and GE Healthcare.

Key Insights and Ideas

  • Hybrid Approach is Key: Combining the non-linear, multivariate learning capabilities of deep learning (CVAE) with the statistically rigorous, shrinkage-based estimation of ComBat offers a powerful synergy for harmonization.
  • Multi-Stage Harmonization: Applying ComBat not just to the raw data but also to the latent space and reconstruction residuals is critical for effectively removing complex batch effects.
  • Robustness through Relaxed Assumptions: DeepComBat's ability to handle imperfect model fits and not demand complete disentanglement of batch and biological information makes it more practical and less prone to introducing synthetic artifacts.
  • Hyperparameter Tuning is Simpler: The paper highlights that DeepComBat's performance is relatively robust to hyperparameter choices, and tuning is guided by interpretable latent space statistics, making it more accessible to users.

Key Results

  • Effective Batch Effect Removal: DeepComBat significantly reduced statistically detectable batch effects, both univariately and multivariately, as evidenced by Anderson-Darling tests and MANOVA. For instance, after harmonization, the mean p-values from Anderson-Darling tests were close to the expected 0.5, indicating distributions across batches were similar.
  • Preservation of Biological Information: DeepComBat maintained or even improved the predictive performance of machine learning models for biological covariates (age, sex, Alzheimer's disease status) compared to raw data and other harmonization methods. For example, AUROC for predicting sex remained high, and accuracy for Alzheimer's disease prediction was comparable to or better than raw data.
  • Outperforms Other Methods: In multivariate ML experiments, DeepComBat consistently achieved the lowest AUROCs for predicting batch status, indicating superior removal of batch information. It also outperformed dcVAE and gcVAE in preserving biological information.
  • Qualitative Visualizations: UMAP and PCA plots showed that DeepComBat effectively clustered subjects by biological factors rather than scanner batch, unlike the raw data.

Limitations and Future Work

  • Computational Cost: While DeepComBat is designed to be relatively fast, it is still slower than purely statistical methods, especially when manual hyperparameter tuning is involved.
  • Sample Size Dependency: As a deep learning model, DeepComBat may require relatively large sample sizes to accurately estimate neural network parameters. For smaller datasets, traditional methods like ComBat might be more suitable.
  • Complexity for Univariate Analyses: For simple univariate analyses, DeepComBat might be overly complex, and standard ComBat could suffice.
  • Potential for Unaccounted Covariates: Information about biological covariates not explicitly conditioned on in DeepComBat might be removed along with batch effects.
  • Cross-Sectional Data Assumption: DeepComBat, like standard ComBat, is designed for cross-sectional data. Extensions to longitudinal data are suggested as future work.

Limitations and Future Work

  • Computational Cost: While DeepComBat is designed to be relatively fast, it is still slower than purely statistical methods, especially when manual hyperparameter tuning is involved.
  • Sample Size Dependency: As a deep learning model, DeepComBat may require relatively large sample sizes to accurately estimate neural network parameters. For smaller datasets, traditional methods like ComBat might be more suitable.
  • Complexity for Univariate Analyses: For simple univariate analyses, DeepComBat might be overly complex, and standard ComBat could suffice.
  • Potential for Unaccounted Covariates: Information about biological covariates not explicitly conditioned on in DeepComBat might be removed along with batch effects.
  • Cross-Sectional Data Assumption: DeepComBat, like standard ComBat, is designed for cross-sectional data. Extensions to longitudinal data are suggested as future work.

Summary updated

Abstract

Neuroimaging data from multiple batches (i.e. acquisition sites, scanner manufacturer, datasets, etc.) are increasingly necessary to gain new insights into the human brain. However, multi-batch data, as well as extracted radiomic features, exhibit pronounced technical artifacts across batches. These batch effects introduce confounding into the data and can obscure biological effects of interest, decreasing the generalizability and reproducibility of findings. This is especially true when multi-batch data is used alongside complex downstream analysis models, such as machine learning methods. Image harmonization methods seeking to remove these batch effects are important for mitigating these issues; however, significant multivariate batch effects remain in the data following harmonization by current state-of-the-art statistical and deep learning methods. We present DeepCombat, a deep learning harmonization method based on a conditional variational autoencoder architecture and the ComBat harmonization model. DeepCombat learns and removes subject-level batch effects by accounting for the multivariate relationships between features. Additionally, DeepComBat relaxes a number of strong assumptions commonly made by previous deep learning harmonization methods and is empirically robust across a wide range of hyperparameter choices. We apply this method to neuroimaging data from a large cognitive-aging cohort and find that DeepCombat outperforms existing methods, as assessed by a battery of machine learning methods, in removing scanner effects from cortical thickness measurements while preserving biological heterogeneity. Additionally, DeepComBat provides a new perspective for statistically-motivated deep learning harmonization methods.
BibTeX
@misc{hu2023deepcombat,
  author = {Fengling Hu and Alfredo Lucas and Andrew A. Chen and Kyle Coleman and Hannah Horng and Raymond W.S. Ng and Nicholas J. Tustison and Kathryn A. Davis and Haochang Shou and Mingyao Li and Russell T. Shinohara and The Alzheimer’s Disease Neuroimaging Initiative},
  title = {{DeepComBat: A Statistically Motivated, Hyperparameter-Robust, Deep Learning Approach to Harmonization of Neuroimaging Data}},
  year = {2023},
  doi = {10.1101/2023.04.24.537396},
  howpublished = {bioRxiv},
}
Figure