Correcting differences in multi-site neuroimaging data using Generative Adversarial Networks

Harrison Nguyen, Richard W. Morris, Anthony Harris, Mayuresh S. Korgoankar, Fábio Ramos

Abstract

Magnetic Resonance Imaging (MRI) of the brain is a critical tool for investigating neurological disorders, yet data acquisition remains expensive and time-consuming. Multi-site studies offer a solution by pooling data to increase statistical power, but images are often confounded by site-specific "fingerprints"—differences in scanner hardware and protocols that introduce bias. We propose a data-driven approach using Generative Adversarial Networks (GANs) to harmonize T1-weighted brain images across different sites. By treating site differences as a style-transfer problem, our model reduces site-specific variance while preserving clinical information related to gender and diagnosis. Our results demonstrate that this "black box" method can normalize imaging sets more effectively than current linear approaches, improving the accuracy of downstream clinical classifications.

Introduction

In neuroimaging research, the pursuit of large datasets often leads to multi-center collaborations. However, every MRI scanner possesses unique characteristics—magnetic field stability, manufacturer-specific gradients (e.g., GE vs. Philips), and acquisition protocols—that create subtle but pervasive variations in the resulting images. These "site effects" act as confounding variables; for instance, if one site primarily scans healthy controls and another scans patients, a machine learning model might inadvertently learn to identify the scanner rather than the pathology.

Traditional normalization methods often rely on physical phantoms or simple linear corrections. However, these methods frequently fail to capture the complex, non-linear nature of how hardware affects biological tissue imaging. This paper explores the use of deep learning to treat scanner-specific artifacts as a "style" that can be translated or removed.

Methods: Cycle-Consistent GANs

The core of our approach is the Cycle-Consistent GAN (CycleGAN) architecture. Unlike standard GANs, CycleGAN is designed for image-to-image translation between two domains without requiring paired examples.

CycleGAN style transfer examples: horse to zebra and apple to orange. This conceptual illustration shows how the architecture learns to map features from one domain to another while maintaining structural integrity.

In this framework, the system uses two generators and two discriminators. Generator $G$ learns to make images from Site A look like they originated from Site B, while Generator $F$ performs the reverse. To ensure the model does not "hallucinate" or delete real biological features, we employ a cycle loss function: if an image is transformed from A to B and then back to A, the resulting image must be identical to the original. This ensures that only the "style" (scanner noise) is modified, while the "content" (brain anatomy) remains intact.

Datasets and Cohort

The study utilized T1-weighted structural MRIs from 313 participants across two distinct sites. Site A used a Philips Achieva 3T scanner, while Site B utilized a GE Discovery MR750 3T scanner. The cohort included healthy controls and patients diagnosed with schizophrenia.

Demographic characteristics of the study participants across Site A and Site B, highlighting the distribution of age, gender, and diagnosis.

For training, the middle five 2D sagittal slices were extracted from preprocessed gray matter density maps. Additionally, a small "ground truth" subset of 11 subjects scanned at both sites was used for direct validation of the transformation accuracy.

Key Results

Our evaluation focused on whether the GAN could remove site bias without losing clinical signal.

  1. Scanner Indistinguishability: Before correction, a Support Vector Machine (SVM) could identify the source scanner with 99.3% accuracy. After GAN correction, this accuracy dropped significantly, indicating that the site-specific "fingerprint" was successfully blurred.
  2. Clinical Classification: Applying GAN correction improved the accuracy of schizophrenia vs. healthy control classification by 3.7% over the baseline. It also outperformed traditional Gaussian Process regression by 1.2%.
  3. Gender Classification: On a balanced dataset, GAN correction improved gender classification accuracy by 15.8% compared to uncorrected data, suggesting that removing scanner noise allows biological signals to emerge more clearly.
  4. Reconstruction Accuracy: For the 11 subjects scanned at both sites, the GAN reduced the Mean Square Error (MSE) between the two different scanner images by 27.02%.

Discussion and Limitations

The primary insight of this work is that hardware artifacts can be treated as a "style" that can be stripped away. By using an unsupervised, non-linear approach, we successfully untangled site effects that linear regression often exacerbated.

However, limitations remain. The current model operates on 2D sagittal slices, which loses volumetric context. Future work will involve moving to 3D convolutional kernels, which, while computationally intensive, would provide a more holistic normalization. Furthermore, while the "black box" nature of the GAN is an advantage for ease of use, further research is needed to interpret exactly which image features (e.g., contrast, edge sharpness) the model is altering during the harmonization process.

Summary updated

Abstract

Magnetic Resonance Imaging (MRI) of the brain has been used to investigate a wide range of neurological disorders, but data acquisition can be expensive, time-consuming, and inconvenient. Multi-site studies present a valuable opportunity to advance research by pooling data in order to increase sensitivity and statistical power. However images derived from MRI are susceptible to both obvious and non-obvious differences between sites which can introduce bias and subject variance, and so reduce statistical power. To rectify these differences, we propose a data driven approach using a deep learning architecture known as generative adversarial networks (GANs). GANs learn to estimate two distributions, and can then be used to transform examples from one distribution into the other distribution. Here we transform T1-weighted brain images collected from two different sites into MR images from the same site. We evaluate whether our model can reduce site-specific differences without loss of information related to gender (male, female) or clinical diagnosis (schizophrenia, bipolar disorder, healthy). When trained appropriately, our model is able to normalise imaging sets to a common scanner set with less information loss compared to current approaches. An important advantage is our method can be treated as a black box that does not require any knowledge of the sources of bias but only needs at least two distinct imaging sets.
BibTeX
@misc{nguyen2018correcting,
  author = {Harrison Nguyen and Richard W. Morris and Anthony Harris and Mayuresh S. Korgoankar and Fábio Ramos},
  title = {{Correcting differences in multi-site neuroimaging data using Generative Adversarial Networks}},
  year = {2018},
  eprint = {1803.09375},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1803.09375},
}
Figure