MISPEL: A supervised deep learning harmonization method for multi-scanner neuroimaging data

Mahbaneh Eshaghzadeh Torbati, Davneet Minhas, Charles M. Laymon, Pauline Maillard, James Wilson, Chang‐Le Chen, Ciprian M. Crainiceanu, Charles DeCarli, Seong Jae Hwang, Dana TudorascubioRxiv 2022 · DOI

TL;DR

This paper introduces MISPEL, a novel deep learning method for harmonizing neuroimaging data from multiple MRI scanners. By preserving anatomical information and learning a "middle-ground" space, MISPEL effectively reduces scanner-induced variability, outperforming existing methods and enabling more reliable large-scale analyses.

What Problem Does This Paper Address?

Aggregating neuroimaging data from various scanners offers immense potential for increased statistical power and robustness in research. However, differences in scanner hardware and acquisition protocols introduce technical variability, often referred to as "scanner effects." This variability can manifest as differences in image contrast, intensity scales, and other image properties, leading to biased downstream analyses and potentially obscuring true biological signals. Existing harmonization methods struggle to address this challenge comprehensively, especially when dealing with more than two scanners or when preserving anatomical details is critical.

Methods

MISPEL is a supervised, multi-scanner harmonization method based on a deep convolutional neural network. It employs a two-step training framework:

  1. Embedding Learning: Encoder-decoder (U-Net) modules are trained for each scanner to reconstruct input images while learning latent embeddings that capture structural information. An "Embedding Coupling" loss encourages these embeddings to be similar across scanners.
  2. Harmonization: The encoders are frozen, and the decoders are trained to synthesize harmonized images. A "Harmonization" loss enforces similarity between harmonized images from different scanners, while a reconstruction loss ensures anatomical preservation.

Crucially, MISPEL maps images to a "middle-ground" harmonized space without requiring a specific target scanner, making it more flexible.

Datasets

The authors introduce a multi-scanner matched dataset of 3T T1-weighted MRI images from 18 participants scanned on four different 3T scanners (GE, Philips, SiemensP, and SiemensT). This dataset is novel and crucial for evaluating harmonization methods by allowing direct comparison of images from the same individual acquired on different scanners.

Key Insights and Ideas

  • The "Middle-Ground" Approach: MISPEL's ability to learn a common harmonized space without a designated target scanner is a significant advantage, particularly when dealing with diverse scanner populations. This avoids the potential bias introduced by selecting a single "best" scanner.
  • Preserving Anatomy is Key: The paper emphasizes the importance of preserving structural (anatomical) information during harmonization. MISPEL's U-Net architecture and carefully designed loss functions are tailored to achieve this, preventing the removal of biologically relevant features.
  • Matched Datasets are Crucial for Evaluation: The introduction of a multi-scanner matched dataset is a valuable contribution, enabling more rigorous assessment of harmonization techniques by directly estimating scanner effects.
  • Harmonization is More Than Just Intensity Normalization: The study highlights that simple intensity normalization methods (like White Stripe) are insufficient for addressing complex scanner effects, underscoring the need for more sophisticated harmonization frameworks.

Key Results

MISPEL consistently outperformed existing methods (White Stripe, RAVEL, and CALAMITI) across various evaluation criteria:

  • Image Similarity: MISPEL significantly improved image similarity and contrast preservation compared to other methods.
  • GM-WM Contrast Similarity: MISPEL achieved the highest overlap in GM-WM contrast, indicating effective harmonization of tissue contrast.
  • Volumetric and Segmentation Similarity: MISPEL demonstrated strong performance in volumetric and segmentation similarity, particularly with the FSL framework, showing minimal bias and variance.
  • Biological Similarity: MISPEL successfully preserved or even enhanced biological signals (SVD group differences) related to Alzheimer's disease biomarkers, outperforming other methods in this crucial aspect.

Limitations and Future Work

  • 2D Network: MISPEL currently uses a 2D network trained on axial slices, which could lead to slice-to-slice inconsistency. The authors suggest future work could involve using a 3D fusion network to address this.
  • Generalizability to Unmatched Data: While MISPEL was trained on matched data, its performance on unmatched multi-scanner data needs further investigation.
  • Potential Bias with Lower-Quality Scanners: If the majority of scanners in a dataset are of lower quality, MISPEL might learn a lower-quality middle-ground space. The authors propose modifying MISPEL to map to a target scanner in such scenarios.
  • Exploration of Other Modalities: Future work will explore MISPEL's performance on other MRI modalities (e.g., FLAIR) and with different degrees of scanner effects.

Summary updated

Abstract

Abstract Large-scale data obtained from aggregation of already collected multi-site neuroimaging datasets has brought benefits such as higher statistical power, reliability, and robustness to the studies. Despite these promises from growth in sample size, substantial technical variability stemming from differences in scanner specifications exists in the aggregated data and could inadvertently bias any downstream analyses on it. Such a challenge calls for data normalization and/or harmonization frameworks, in addition to comprehensive criteria to estimate the scanner-related variability and evaluate the harmonization frameworks. In this study, we propose MISPEL (Multi-scanner Image harmonization via Structure Preserving Embedding Learning), a supervised multi-scanner harmonization method that is naturally extendable to more than two scanners. We also designed a set of criteria to investigate the scanner-related technical variability and evaluate the harmonization techniques. As an essential requirement of our criteria, we introduced a multi-scanner matched dataset of 3T T1 images across four scanners, which, to the best of our knowledge is one of the few datasets of this kind. We also investigated our evaluations using two popular segmentation frameworks: FSL and segmentation in statistical parametric mapping (SPM). Lastly, we compared MISPEL to popular methods of normalization and harmonization, namely White Stripe, RAVEL, and CALAMITI. MISPEL outperformed these methods and is promising for many other neuroimaging modalities.
BibTeX
@misc{torbati2022mispel,
  author = {Mahbaneh Eshaghzadeh Torbati and Davneet Minhas and Charles M. Laymon and Pauline Maillard and James Wilson and Chang‐Le Chen and Ciprian M. Crainiceanu and Charles DeCarli and Seong Jae Hwang and Dana Tudorascu},
  title = {{MISPEL: A supervised deep learning harmonization method for multi-scanner neuroimaging data}},
  year = {2022},
  doi = {10.1101/2022.07.27.501786},
  howpublished = {bioRxiv},
}
Figure