Neuroharmony: A new tool for harmonizing volumetric MRI data from unseen scanners

Vince D. Calhoun, Rafael Garcia-Dias, Cristina Scarpazza, Lea Baecker, Sandra Vieira, Walter H. L. Pinaya, Aiden P. Corvin, Alberto Redolfi, Barnaby Nelson, Benedicto Crespo-Facorro, Colm McDonald, Diana Tordesillas-Gutiérrez, Dara M. Cannon, David Mothersill, Dennis Hernaus, Derek W. Morris, Esther Setien-Suero, Gary Donohoe, Giovanni B. Frisoni, Andrea MechelliNeuroImage 2020 · DOI

TL;DR

This paper introduces Neuroharmony, a novel tool designed to harmonize MRI data from different scanners, even those not encountered during training. This is crucial for building robust machine learning models in neuroimaging that can generalize to real-world clinical settings where scanner variability is a major hurdle.

What Problem Does This Paper Address?

The increasing availability of MRI datasets has fueled interest in applying machine learning to neuroimaging for diagnosing and understanding brain disorders. However, a significant challenge is the heterogeneity introduced by data collected from different MRI scanners and acquisition protocols. This variability can lead to inconsistent measurements of brain region volumes, making it difficult to train reliable models that generalize to new, unseen data. Existing harmonization methods, like ComBat, often require a statistically representative sample from each scanner, which is impractical for clinical translation where models must work with individual scans from unknown scanners. This paper aims to bridge this gap by developing a tool that can harmonize data from unseen scanners without needing such representative samples.

Methods

The core of Neuroharmony lies in its ability to predict harmonization corrections based on image quality metrics (IQMs) extracted directly from individual MRI scans. The authors first used the established ComBat method on a large dataset (15,026 subjects from 62 scanners) to generate "ground truth" harmonization corrections. They then trained a machine learning model (specifically, random forest regressors) to learn the relationship between various IQMs (e.g., contrast-to-noise ratio, signal-to-noise ratio, spatial resolution metrics) and the ComBat-derived corrections for each region of interest (ROI). This trained model, Neuroharmony, can then take IQMs from a new, unseen scan and predict the necessary correction to harmonize its volumetric data. The authors employed a leave-one-scanner-out cross-validation strategy for hyperparameter tuning and used principal component analysis (PCA) to reduce the dimensionality of the input features.

Datasets

The study utilized a large mega-dataset comprising 15,026 T1-weighted MRI scans from 62 different scanners for training and evaluating the ComBat harmonization and subsequently training Neuroharmony. For validation, an independent set of 454 subjects from 16 scanners was used. The paper also references several publicly available datasets that contributed to the training data, including ABIDEII, ADHD200, BIOBANK, COBRE, and PPMI, among others.

Key Insights and Ideas

  • IQMs as a proxy for scanner bias: The fundamental insight is that intrinsic image quality metrics, extractable from individual scans, can effectively capture and predict scanner-related biases. This bypasses the need for large, representative samples from each scanner.
  • Generalization to unseen scanners: Neuroharmony's ability to predict harmonization corrections for scanners not included in its training set is a significant advancement for clinical translation.
  • ComBat's limitations: While ComBat is effective, it struggles with certain types of distortions, like double-peak distributions in ROI volumes, which Neuroharmony's IQM-based approach might implicitly handle better by learning from a broader range of image characteristics.
  • Regional variability in harmonization: The effectiveness of harmonization varies across different brain regions, with some (like ventricles) showing higher natural variability and being more challenging to correct.

Key Results

  • ComBat harmonization successfully reduced scanner-related bias across most ROIs, raising p-values from the Kolmogorov-Smirnov (K-S) test above the 0.001 threshold for nearly all scanner pairs.
  • Neuroharmony demonstrated effectiveness in harmonizing data from unseen scanners, achieving p-values above 0.001 for most ROIs in the validation set when compared to ComBat-harmonized training data.
  • For grey matter, white matter, and the whole brain, Neuroharmony achieved p-values of 0.455, 0.667, and 0.803, respectively, indicating successful harmonization at these broader levels.
  • The tool was more accurate in correcting regions with lower natural variability and less so for regions with high variability (e.g., lateral ventricles).

Limitations and Future Work

  • Coverage of scanner configurations: The authors acknowledge that while their training sample was diverse, it cannot guarantee coverage of all possible scanner configurations and acquisition protocols. If a scanner's IQMs fall outside the training range, Neuroharmony's performance might be suboptimal.
  • Regional performance variation: Neuroharmony's accuracy varies by brain region, with some regions (e.g., lateral ventricles) being more challenging to harmonize due to high intrinsic variability.
  • Clinical translation: The current validation is primarily on healthy subjects. Further work is needed to assess Neuroharmony's effectiveness in harmonizing data from patient populations.
  • Preserving variance: While the current approach removes variance due to age and sex, future work could explore options to preserve this variance if it's clinically relevant.
  • Understanding complex distortions: Further investigation is needed to understand the nature of distortions that ComBat struggles with (e.g., double peaks) and how Neuroharmony addresses them.

Summary updated

BibTeX
@article{calhoun2020neuroharmony,
  author = {Vince D. Calhoun and Rafael Garcia-Dias and Cristina Scarpazza and Lea Baecker and Sandra Vieira and Walter H. L. Pinaya and Aiden P. Corvin and Alberto Redolfi and Barnaby Nelson and Benedicto Crespo-Facorro and Colm McDonald and Diana Tordesillas-Gutiérrez and Dara M. Cannon and David Mothersill and Dennis Hernaus and Derek W. Morris and Esther Setien-Suero and Gary Donohoe and Giovanni B. Frisoni and Andrea Mechelli},
  title = {{Neuroharmony: A new tool for harmonizing volumetric MRI data from unseen scanners}},
  journal = {NeuroImage},
  year = {2020},
  doi = {10.1016/j.neuroimage.2020.117127},
}
Figure