Data augmentation for deep learning based accelerated MRI reconstruction

Zalan Fabian, Reinhard Heckel, Mahdi SoltanolkotabiOpenReview 2020 · OpenReview

TL;DR

This paper introduces MRAugment, a novel data augmentation pipeline specifically designed for accelerated MRI reconstruction. By carefully considering the physics of MRI acquisition, MRAugment significantly reduces the amount of training data needed to achieve state-of-the-art reconstruction quality, making deep learning more accessible for this critical medical imaging technique.

What Problem Does This Paper Address?

Accelerated Magnetic Resonance Imaging (MRI) is crucial for reducing scan times, improving patient comfort, and increasing accessibility. However, achieving high-quality reconstructions often relies on deep learning models that typically require vast amounts of training data. Collecting such large, diverse datasets for medical imaging is challenging due to cost, patient privacy, and data compatibility issues. This paper addresses the problem of reducing the data dependency of deep learning models for accelerated MRI reconstruction by proposing an effective data augmentation strategy.

Methods

The core contribution is the MRAugment pipeline. Unlike standard data augmentation in classification (where labels remain unchanged after augmentation), MRI reconstruction requires generating both an augmented target image and its corresponding undersampled measurements. MRAugment tackles this by:

  • Physics-Informed Augmentation: It specifically accounts for the relationship between image data and k-space measurements, ensuring that augmented data is physically plausible.
  • Individual Coil Augmentation: It augments individual coil images before combining them, which is crucial for parallel MRI. This approach preserves the noise statistics of the measurements, a critical factor for generalization.
  • Transformations: It employs two types of transformations:
  • Pixel-preserving: Flips, translations, and 90-degree rotations.
  • General affine: Arbitrary translations, rotations, scaling, and shearing.
  • Augmentation Scheduling: It proposes a schedule where augmentation probability increases over training epochs, starting from zero, to optimize learning.

Datasets

The authors evaluated MRAugment on the following datasets:

  • Stanford 2D FSE dataset: A public dataset of 89 fully-sampled MRI volumes.
  • Stanford Fullysampled 3D FSE Knees dataset: A public dataset of 20 fully-sampled k-space volumes of knees.
  • fastMRI dataset: A large, publicly available dataset of knee and brain MRI volumes, used extensively for benchmarking.

Key Insights and Ideas

  • Physics Matters for DA: Naive data augmentation strategies that ignore the underlying physics of MRI (e.g., augmenting real-valued images without considering k-space) lead to poor performance due to noise distribution mismatches.
  • Noise Preservation is Key: Augmenting complex-valued coil images directly and ensuring noise statistics are preserved is critical for successful generalization.
  • Low-Data Regime is Where DA Shines: MRAugment provides the most significant benefits in low-data scenarios, enabling state-of-the-art performance with a fraction of the data.
  • Complementary Augmentations: Combining different types of augmentations (pixel-preserving and interpolating) yields better results than using either alone.
  • Adaptive Scheduling is Beneficial: Gradually increasing augmentation probability during training is more effective than a constant probability.

Key Results

  • Significant Data Reduction: For 8-fold acceleration on the fastMRI knee dataset, MRAugment achieved state-of-the-art SSIM with only 10% of the training data for multi-coil reconstruction and 33% for single-coil reconstruction.
  • Improved Reconstruction Quality: In low-data regimes, MRAugment significantly boosted reconstruction SSIM and visual quality, recovering fine details often missed by models without augmentation.
  • Robustness: MRAugment improved generalization to unseen MRI scanners and anatomies, and helped prevent hallucinations caused by overfitting.
  • Comparable to State-of-the-Art: Even in high-data regimes, MRAugment maintained performance comparable to models trained on full datasets.

Limitations and Future Work

  • Diminishing Returns in High-Data Regimes: While MRAugment doesn't degrade performance in high-data regimes, its benefits diminish compared to low-data scenarios. The authors suggest that combining DA with larger models might unlock further gains.
  • Learning Augmentation Strategies: The paper uses a predefined set of augmentations. Future work could explore learning optimal augmentation strategies automatically from data.
  • Adaptive Augmentation Scheduling: While exponential scheduling was found effective, developing an adaptive scheme that automatically adjusts augmentation strength could further improve results.
  • Generative Frameworks: Combining MRAugment with generative models to synthesize fully-sampled MRI data from noisy measurements is a promising future direction.

Summary updated

Abstract

Deep neural networks have emerged as very successful tools for image restoration and reconstruction tasks. These networks are often trained end-to-end to directly reconstruct an image from a noisy or corrupted measurement of that image. To achieve state-of-the-art performance, training on large and diverse sets of images is considered critical. However, it is often difficult and/or expensive to collect large amounts of training images. Inspired by the success of Data Augmentation (DA) for classification problems, in this paper, we propose a pipeline for data augmentation for image reconstruction tasks arising in medical imaging and explore its effectiveness at reducing the required training data in a variety of settings. We focus on accelerated magnetic resonance imaging, where the goal is to reconstruct an image from a few under-sampled linear measurements. Our DA pipeline is specifically designed to utilize the invariances present in medical imaging measurements as naive DA strategies that neglect the physics of the problem fail. We demonstrate the effectiveness of our data augmentation pipeline by showing that for some problem regimes, DA can achieve comparable performance to the state of the art on the FastMRI dataset while using significantly fewer training data. Specifically, for 8-fold acceleration we achieve performance comparable to the state of the art with only $10\%$ of the training data for multi-coil reconstruction and with only $33\%$ of the training data for single-coil reconstruction. Our findings show that in the low-data regime DA is beneficial, whereas in the high-data regime it has diminishing returns.
BibTeX
@misc{fabian2020data,
  author = {Zalan Fabian and Reinhard Heckel and Mahdi Soltanolkotabi},
  title = {{Data augmentation for deep learning based accelerated MRI reconstruction}},
  year = {2020},
  url = {https://openreview.net/forum?id=lH2ukHnGDdq},
  howpublished = {OpenReview},
}
Figure