Learning Deep MRI Reconstruction Models from Scratch in Low-Data Regimes

Salman Uh Dar, Şaban Öztürk, Muzaffer Özbey, Tolga ÇukurarXiv 2023 · DOI · arXiv

TL;DR

PSFNet is a novel deep learning architecture for MRI reconstruction that excels in "low-data regimes," where large training datasets are unavailable. By fusing scan-specific (subject-driven) and scan-general (data-driven) information in a parallel-stream architecture, it achieves high-quality image recovery with an order of magnitude less training data than standard models and significantly faster inference than traditional subject-specific methods.

What Problem Does This Paper Address?

Magnetic Resonance Imaging (MRI) is notoriously slow because it collects data in "k-space" (the spatial frequency domain). To speed up scans, clinicians often "undersample" this data, but reconstructing a clear image from a partial dataset is an ill-posed inverse problem that usually results in aliasing artifacts (blurring or ghosting).

Current deep learning solutions generally fall into two camps:

  1. Scan-General (SG) Priors: These models (like MoDL) are pre-trained on hundreds of subjects. They are fast at test time but require massive datasets and often struggle to generalize to "out-of-distribution" patients (e.g., different anatomy or different scanner settings).
  2. Scan-Specific (SS) Priors: These models (like RAKI) learn a reconstruction specifically for the patient currently in the scanner. They don't need a training set, but they are computationally expensive and slow during the actual clinical exam.

Furthermore, most state-of-the-art models use algorithmic unrolling, which stacks these priors in a serial chain. If the model is trained on a small dataset, errors from the first "link" in the chain propagate and amplify, leading to poor final images.

Methods

The authors introduce PSFNet (Parallel-Stream Fusion Model). Its core innovation is moving away from serial chains in favor of a parallel-stream architecture:

  • The SS Stream: Uses a linear kernel learned from the "calibration region" (the fully sampled center of k-space) of the specific test scan. This ensures the model respects the physics of that specific acquisition.
  • The SG Stream: Uses a nonlinear Convolutional Neural Network (CNN) to capture complex image features learned across a training population.
  • Learnable Fusion: Instead of one following the other, the outputs of both streams are combined using learnable weighting parameters ($\eta$ and $\gamma$). This allows the network to adaptively "trust" one stream more than the other depending on how much training data is available.
  • Cascaded Design: This parallel fusion is repeated across multiple stages (cascades), each followed by a Data Consistency (DC) block to ensure the reconstruction remains faithful to the actual acquired k-space data.

Datasets

The model was evaluated using the NYU fastMRI database, specifically focusing on:

  • Brain MRI: Contrast-enhanced T1-weighted (cT1) and T2-weighted scans.
  • Undersampling: Retrospectively undersampled at 4x and 8x acceleration factors.
  • Data Regimes: The authors systematically tested the model with training sets ranging from as few as 2 samples up to 500 samples to demonstrate its efficiency.

Key Insights and Ideas

  • Parallel vs. Serial: The "aha moment" is that parallel fusion acts as a safety net. In low-data regimes, a serial SG prior might produce "garbage" that the rest of the network can't fix. In a parallel setup, the network can learn to rely on the simpler, more stable SS prior while the SG prior is still "finding its feet" during training.
  • Linear for Speed, Nonlinear for Power: By keeping the scan-specific part linear, the authors maintain fast inference times (comparable to standard SG models) while still gaining the generalization benefits of subject-specific learning.
  • Generalization Power: Because PSFNet incorporates the physical signal model of the specific scan via the SS stream, it is much more resilient to changes in tissue contrast or sampling patterns than "pure" deep learning models.

Key Results

  • Data Efficiency: PSFNet trained on just 10–30 samples outperformed standard SG models (MoDL) trained on the same amount. It even surpassed the performance of traditional SS methods that require no training data.
  • Generalization: When trained on T1 images and tested on T2 images (domain transfer), PSFNet maintained high performance, whereas standard models saw significant drops in image quality.
  • Speed: PSFNet enables inference in approximately 1.13 seconds per subject. This is an order of magnitude faster than nonlinear SS methods like sRAKI-RNN (~285 seconds) or SPARK (~23 seconds).
  • Unsupervised Success: The architecture also works in an unsupervised setting (PSFNetUS), outperforming other unsupervised methods in low-data scenarios.

Limitations and Future Work

  • Linear SS Prior: The current SS stream is linear to save time. The authors suggest that incorporating nonlinear SS priors could improve quality further, though it would increase the computational burden.
  • High-Frequency Details: While PSFNet is excellent at removing artifacts, the authors note that adding an adversarial loss (from a GAN) could help recover even finer structural details.
  • Architectural Evolution: Future versions could explore replacing the CNN blocks with Transformers or incorporating multimodal data (fusing different types of MRI scans) to further boost performance.

Summary updated

Abstract

Magnetic resonance imaging (MRI) is an essential diagnostic tool that suffers from prolonged scan times. Reconstruction methods can alleviate this limitation by recovering clinically usable images from accelerated acquisitions. In particular, learning-based methods promise performance leaps by employing deep neural networks as data-driven priors. A powerful approach uses scan-specific (SS) priors that leverage information regarding the underlying physical signal model for reconstruction. SS priors are learned on each individual test scan without the need for a training dataset, albeit they suffer from computationally burdening inference with nonlinear networks. An alternative approach uses scan-general (SG) priors that instead leverage information regarding the latent features of MRI images for reconstruction. SG priors are frozen at test time for efficiency, albeit they require learning from a large training dataset. Here, we introduce a novel parallel-stream fusion model (PSFNet) that synergistically fuses SS and SG priors for performant MRI reconstruction in low-data regimes, while maintaining competitive inference times to SG methods. PSFNet implements its SG prior based on a nonlinear network, yet it forms its SS prior based on a linear network to maintain efficiency. A pervasive framework for combining multiple priors in MRI reconstruction is algorithmic unrolling that uses serially alternated projections, causing error propagation under low-data regimes. To alleviate error propagation, PSFNet combines its SS and SG priors via a novel parallel-stream architecture with learnable fusion parameters. Demonstrations are performed on multi-coil brain MRI for varying amounts of training data. PSFNet outperforms SG methods in low-data regimes, and surpasses SS methods with few tens of training samples.
BibTeX
@misc{dar2023learning,
  author = {Salman UH Dar and Şaban Öztürk and Muzaffer Özbey and Tolga Çukur},
  title = {{Learning Deep MRI Reconstruction Models from Scratch in Low-Data Regimes}},
  year = {2023},
  eprint = {2301.02613},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2301.02613},
}
Figure