Contrastive Learning MRI Reconstruction

Mevan Ekanayake, Zhifeng Chen, Gary F. Egan, Mehrtash Harandi, Zhaolin ChenarXiv 2023 · OpenReview

TL;DR

This paper introduces CL-MRI, a self-supervised contrastive learning pretraining method that significantly improves the accuracy of deep learning-based MRI reconstruction from undersampled data. By learning a richer latent representation of MRI images, CL-MRI enhances reconstruction quality, robustness to various challenges, and even enables transfer learning across different anatomical regions.

What Problem Does This Paper Address?

Magnetic Resonance Imaging (MRI) is a powerful diagnostic tool, but its long scan times lead to patient discomfort and motion artifacts. To speed up scans, MRI data is often undersampled in k-space, which severely degrades image quality, introducing aliasing artifacts and noise. While deep learning (DL) methods have advanced MRI reconstruction, they still struggle to meet the high standards required for clinical diagnosis, especially at higher acceleration factors. This paper addresses the need for more accurate and robust DL-based MRI reconstruction techniques that can effectively handle undersampled data and its inherent quality issues.

Methods

The core innovation is CL-MRI, a self-supervised contrastive learning pretraining framework. Instead of directly reconstructing from undersampled images, CL-MRI first trains a feature extraction model to map undersampled MRI images (from the same scan but with different acceleration factors) into a latent space. This contrastive learning process aims to:

  • Maximize Mutual Information: By treating different undersampled versions of the same scan as positive pairs, the model learns to preserve and enhance the shared information.
  • Optimize Information Content: This process effectively "pulls" representations of similar data points closer in the latent space and "pushes" dissimilar ones apart, creating a more informative and robust representation.
  • Downstream Reconstruction: These enhanced latent representations then serve as input to a standard DL reconstruction model (like U-Net, D5C5, MICCAN, ReconFormer), which is then trained with a reconstruction loss.

This pretraining step is self-supervised, meaning it doesn't require fully sampled ground truth images, which are often difficult to obtain. The paper also analyzes the properties of the learned latent space, focusing on Alignment (closeness of representations from positive pairs) and Uniformity (distribution of representations in the latent space).

Datasets

The study utilized the following datasets:

  • fastMRI brain dataset: Used for training and validation of the CL-MRI framework and for evaluating reconstruction models. This dataset contains multi-coil FLAIR brain MRI data from 3T and 1.5T scanners.
  • fastMRI knee dataset: Used for out-of-distribution assessments. This dataset comprises multi-coil proton density-weighted (PD) knee MRI data from 3T and 1.5T scanners.

Key Insights and Ideas

  • Latent Space Matters: The paper strongly suggests that the quality of the latent representation fed into the reconstruction model is crucial. CL-MRI effectively creates a richer, more informative latent space that significantly benefits downstream reconstruction.
  • Self-Supervision for MRI: Demonstrates the power of self-supervised learning in MRI reconstruction, overcoming the reliance on scarce fully sampled data.
  • Transfer Learning Powerhouse: CL-MRI's feature extractor, trained on brain MRI, can be effectively used for knee MRI reconstruction without retraining, highlighting its generalizability and addressing data scarcity issues.
  • Robustness Under Fire: The CL-MRI framework shows remarkable resilience to various challenges, including different k-space sampling patterns, measurement noise, and even pathological abnormalities in the MRI scans.
  • Alignment and Uniformity Boost Performance: The analysis of the latent space reveals that CL-MRI promotes desirable properties like alignment and uniformity, which are linked to improved downstream task accuracy.

Key Results

  • Improved Accuracy: CL-MRI consistently improved reconstruction accuracy (measured by NMSE, PSNR, and SSIM) across various DL reconstruction models (D5C5, U-Net, MICCAN, ReconFormer) and acceleration factors (4X to 12X). For example, at 12X acceleration, the performance gain for D5C5 was significantly larger compared to lower accelerations.
  • Qualitative Enhancements: Reconstructed images with CL-MRI showed reduced artifacts, sharper contrasts, and more intricate details, particularly noticeable at higher acceleration factors.
  • Out-of-Distribution Success: The framework demonstrated successful transfer learning, with a feature extractor trained on brain MRI data improving knee MRI reconstruction performance.
  • Robustness: CL-MRI showed minimal performance degradation when faced with different sampling masks, varying levels of measurement noise (down to 20 dB SNR), and pathological abnormalities.
  • Faster Convergence: CL-MRI pretraining led to faster training convergence for the downstream reconstruction models.

Limitations and Future Work

  • Memory Requirements: The contrastive learning pretraining phase requires significant GPU memory, especially with multiple acceleration factors, as the batch size increases proportionally.
  • Additional Computation: The inference phase involves an extra step for generating contrastive features, adding computational overhead. However, the authors consider this a reasonable tradeoff for the accuracy gains.
  • Future Directions: The authors plan to explore more advanced contrastive learning algorithms (e.g., Swav, MoCo, DINO, BYOL, Barlow Twins) to potentially achieve even greater enhancements in undersampled MRI reconstruction.

Summary updated

Abstract

In Magnetic Resonance Imaging (MRI), image acquisitions are often undersampled in the measurement domain to accelerate the scanning process, at the expense of image quality. However, image quality is a crucial factor that influences the accuracy of clinical diagnosis; hence, high-quality image reconstruction from undersampled measurements has been a key area of research. Recently, deep learning (DL) methods have emerged as the state-of-the-art for MRI reconstruction, typically involving deep neural networks to transform undersampled MRI images into high-quality MRI images through data-driven processes. Nevertheless, there is clear and significant room for improvement in undersampled DL MRI reconstruction to meet the high standards required for clinical diagnosis, in terms of eliminating aliasing artifacts and reducing image noise. In this paper, we introduce a self-supervised pretraining procedure using contrastive learning to improve the accuracy of undersampled DL MRI reconstruction. We use contrastive learning to transform the MRI image representations into a latent space that maximizes mutual information among different undersampled representations and optimizes the information content at the input of the downstream DL reconstruction models. Our experiments demonstrate improved reconstruction accuracy across a range of acceleration factors and datasets, both quantitatively and qualitatively. Furthermore, our extended experiments validate the proposed framework's robustness under adversarial conditions, such as measurement noise, different k-space sampling patterns, and pathological abnormalities, and also prove the transfer learning capabilities on MRI datasets with completely different anatomy. Additionally, we conducted experiments to visualize and analyze the properties of the proposed MRI contrastive learning latent space.
BibTeX
@misc{ekanayake2023contrastive,
  author = {Mevan Ekanayake and Zhifeng Chen and Gary F. Egan and Mehrtash Harandi and Zhaolin Chen},
  title = {{Contrastive Learning MRI Reconstruction}},
  year = {2023},
  eprint = {2306.00530},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2306.00530},
}
Figure