Removal of Scanner Effects in Covariance Improves Multivariate Pattern Analysis in Neuroimaging Data

Andrew A. Chen, Joanne C. Beer, Nicholas J. Tustison, Philip A. Cook, Russell T. Shinohara, Haochang Shou, the Alzheimer’s Disease Neuroimaging InitiativebioRxiv 2019 · DOI

TL;DR

Neuroimaging studies often suffer from scanner-induced variability that can obscure important biological signals. While existing methods harmonize mean and variance, they miss crucial covariance differences. This paper introduces "CovBat," a novel method that harmonizes covariance, significantly improving multivariate pattern analysis (MVPA) and enabling more reliable detection of biological associations in multi-site neuroimaging data.

What Problem Does This Paper Address?

Neuroimaging research increasingly relies on multi-site studies to achieve larger sample sizes and improve the generalizability of findings. However, data acquired from different scanners can exhibit significant variability, often referred to as "scanner effects." These effects can bias comparisons, mask true biological associations, and even introduce spurious ones. Existing harmonization techniques primarily focus on correcting differences in the mean and variance of measurements across scanners. This paper argues that these methods are insufficient for advanced analyses like Multivariate Pattern Analysis (MVPA), which leverage the complex relationships (covariance) between different brain regions. The authors demonstrate that scanner effects can profoundly impact these covariance structures, and that standard harmonization methods fail to address this critical issue, potentially leading to inaccurate conclusions.

Methods

The core contribution of this paper is the development of CovBat (Correcting Covariance Batch Effects). This method builds upon the existing ComBat framework, which addresses mean and variance batch effects. CovBat's key innovation is to extend harmonization to the covariance structure of the data.

Here's a breakdown of the approach:

  • Step 1: ComBat Adjustment: First, the standard ComBat algorithm is applied to remove mean and variance shifts in the marginal distributions of the neuroimaging measures (cortical thickness in this case).
  • Step 2: Principal Component Analysis (PCA): PCA is performed on the ComBat-adjusted residuals to identify the principal components (PCs) that capture the most variance in the data. The idea is that scanner effects might be more prominent in the variance of these principal component scores.
  • Step 3: Covariance Harmonization in PC Space: The method then aims to harmonize the covariance of these PC scores across scanners. It models the within-scanner covariance of PC scores and adjusts them to match the pooled covariance structure estimated from all data. This is analogous to how ComBat adjusts means and variances.
  • Step 4: Reconstruction: Finally, the harmonized PC scores are projected back into the original feature space, and the previously estimated intercepts and covariates are added back to obtain the CovBat-adjusted observations.

The authors also employ Multivariate Pattern Analysis (MVPA), specifically using random forests, to evaluate the effectiveness of their harmonization method. This involves training models to predict scanner manufacturer, sex, and Alzheimer's disease status.

Datasets

The primary dataset used in this study is the Alzheimer's Disease Neuroimaging Initiative (ADNI). This is a well-established, multi-site dataset commonly used in neuroimaging research. The authors specifically use cortical thickness measurements derived from T1-weighted MRI scans from ADNI participants.

Key Insights and Ideas

  • Covariance is Crucial for MVPA: The paper strongly emphasizes that MVPA, which leverages the relationships between brain regions, is highly sensitive to scanner-induced differences in covariance. Standard harmonization methods that only address mean and variance are insufficient.
  • Scanner Effects Manifest in Covariance: The study provides compelling evidence that significant differences in covariance structures exist across scanners, even for the same type of neuroimaging measure (cortical thickness).
  • CovBat Effectively Harmonizes Covariance: The proposed CovBat method demonstrates a significant reduction in inter-scanner covariance differences, leading to improved MVPA performance.
  • Harmonization Can "Hide" Scanner Identity: A key finding is that CovBat effectively removes scanner-specific information to the point where machine learning models can no longer reliably predict the scanner manufacturer, suggesting successful harmonization.
  • Biological Associations are Preserved: Crucially, CovBat not only removes scanner effects but also preserves biologically meaningful associations (e.g., between brain structure and sex or disease status), which is essential for the clinical utility of neuroimaging data.

Key Results

  • Visual Evidence: Figure 2 clearly shows that unharmonized covariance matrices differ substantially across scanners. ComBat shows modest improvements, while CovBat dramatically reduces these inter-scanner differences.
  • Quantitative Distance Reduction: Table 2 quantifies this improvement, showing significantly smaller pairwise Frobenius distances between scanner covariance matrices after CovBat harmonization compared to unharmonized or ComBat-adjusted data.
  • Scanner Detection Reduced: In MVPA experiments (Figure 3a), the Area Under the Curve (AUC) for detecting scanner manufacturer drops from 0.89 (unharmonized) and 0.66 (ComBat) to near chance levels (0.46) after CovBat.
  • Biological Associations Maintained: For detecting Alzheimer's Disease (AD), AUC increases from 0.74 (unharmonized) to 0.78 (ComBat) and 0.79 (CovBat). Similarly, for detecting sex, AUC increases from 0.66 to 0.69 and 0.70, respectively. These improvements and the lack of significant differences between ComBat and CovBat for biological associations suggest CovBat preserves important signals.
  • Simulation Performance: Simulation studies (Figures 4, 5, 6 and Tables 4, 5) consistently show that CovBat outperforms ComBat in recovering true covariance structures and reducing site effects in MVPA, especially when covariance effects are present.

Limitations and Future Work

  • Covariate Specification: A limitation of CovBat is that it is a covariate-assisted harmonization method, requiring the explicit specification of covariates to protect. If important covariates are not included, their effects might be removed alongside scanner effects.
  • Potential for Spurious Associations: The authors caution that, like ComBat, CovBat could potentially introduce spurious associations if not carefully applied, especially in unbalanced study designs. They recommend performing analyses both with and without harmonization.
  • High-Dimensional Settings: In simulation studies, CovBat's performance can be affected in high-dimensional settings (many features, small sample sizes), potentially leading to increased scanner detection due to poor covariance estimation.
  • Future Work: The authors suggest exploring extensions to CovBat that can model more complex scanner effects, potentially using spectral models or projection methods. They also recommend further investigation into how specific scanner properties influence covariance structures and the application of CovBat to other multi-site neuroimaging studies and modalities (e.g., functional, metabolic). They also highlight the need to assess the susceptibility of genomic data to similar biases.

Summary updated

Abstract

Abstract To acquire larger samples for answering complex questions in neuroscience, researchers have increasingly turned to multi-site neuroimaging studies. However, these studies are hindered by differences in images acquired across multiple scanners. These effects have been shown to bias comparison between scanners, mask biologically meaningful associations, and even introduce spurious associations. To address this, the field has focused on harmonizing data by removing scanner-related effects in the mean and variance of measurements. Contemporaneously with the increase in popularity of multi-center imaging, the use of multivariate pattern analysis (MVPA) has also become commonplace. These approaches have been shown to provide improved sensitivity, specificity, and power due to their modeling the joint relationship across measurements in the brain. In this work, we demonstrate that methods for removing scanner effects in mean and variance may not be sufficient for MVPA. This stems from the fact that such methods fail to address how correlations between measurements can vary across scanners. Data from the Alzheimer’s Disease Neuroimaging Initiative is used to show that considerable differences in covariance exist across scanners and that popular harmonization techniques do not address this issue. We also propose a novel methodology that harmonizes covariance of multivariate image measurements across scanners and demonstrate its improved performance in data harmonization.
BibTeX
@misc{chen2019removal,
  author = {Andrew A. Chen and Joanne C. Beer and Nicholas J. Tustison and Philip A. Cook and Russell T. Shinohara and Haochang Shou and the Alzheimer’s Disease Neuroimaging Initiative},
  title = {{Removal of Scanner Effects in Covariance Improves Multivariate Pattern Analysis in Neuroimaging Data}},
  year = {2019},
  doi = {10.1101/858415},
  howpublished = {bioRxiv},
}
Figure