Zero-Shot Self-Supervised Learning for MRI Reconstruction

Burhaneddin Yaman, Seyed Amir Hossein Hosseini, Mehmet Akçakaya

Abstract

Deep learning (DL) has emerged as a powerful tool for accelerated MRI reconstruction, but often necessitates a database of fully-sampled measurements for training. Recent self-supervised and unsupervised learning approaches enable training without fully-sampled data. However, a database of undersampled measurements may not be available in many scenarios, especially for scans involving contrast or translational acquisitions in development. Moreover, recent studies show that database-trained models may not generalize well when the unseen measurements differ in terms of sampling pattern, acceleration rate, SNR, image contrast, and anatomy. Such challenges necessitate a new methodology to enable subject-specific DL MRI reconstruction without external training datasets, since it is clinically imperative to provide high-quality reconstructions that can be used to identify lesions/disease for every individual. In this work, we propose a zero-shot self-supervised learning approach to perform subject-specific accelerated DL MRI reconstruction to tackle these issues. The proposed approach partitions the available measurements from a single scan into three disjoint sets. Two of these sets are used to enforce data consistency and define loss during training for self-supervision, while the last set serves to self-validate, establishing an early stopping criterion. In the presence of models pre-trained on a database with different image characteristics, we show that the proposed approach can be combined with transfer learning for faster convergence time and reduced computational complexity.

Introduction

Deep learning (DL) has revolutionized accelerated MRI by "filling in the blanks" of undersampled data, significantly shortening scan times. However, standard DL approaches suffer from two major bottlenecks: data scarcity and the generalization gap. They typically require large databases of "fully-sampled" (ground truth) MRI scans, which are difficult to collect for rare clinical protocols. Furthermore, models trained on specific databases often perform poorly when applied to different anatomies, noise levels, or sampling masks. In a clinical setting, a reconstruction failure can lead to misdiagnosis. This work introduces ZS-SSL, a methodology to create a "subject-specific" model that learns to reconstruct an image using only the measurements acquired from that specific patient during a single session.

Methods

The core innovation is a zero-shot self-supervised learning (ZS-SSL) framework built on Physics-Guided Deep Learning Reconstruction (PG-DLR). Standard PG-DLR "unrolls" an optimization algorithm into a neural network that alternates between data consistency (DC) steps and regularization steps.

Figure 1: Overview of the ZS-SSL framework and the unrolled network architecture. The framework partitions k-space data into training, loss, and validation sets to enable subject-specific learning.

ZS-SSL adapts this by partitioning the available k-space measurements ($\Omega$) from a single scan into three disjoint sets:

  • Training Set ($\Theta$): Used in the DC units of the network.
  • Loss Set ($\Lambda$): Used to define the self-supervision loss by comparing the network's output to these held-out measurements.
  • Validation Set ($\Gamma$): This acts as an automated early stopping criterion. Because training on a single image is prone to overfitting, the network monitors the loss on this third set. When the validation loss starts to rise (forming an "L-curve"), training stops to prevent the model from fitting to noise.

The authors also propose ZS-SSL-TL, which utilizes transfer learning. By initializing the ZS-SSL model with weights from a model pre-trained on a different database, the time required for the model to converge is significantly reduced.

Key Insights and Results

Unlike database-trained models that learn a general "prior," ZS-SSL learns the specific characteristics of the scan currently in the scanner. This makes it inherently agnostic to changes in sampling patterns or SNR. The addition of the third validation set ($\Gamma$) allows for an automated stopping criterion, removing the need for manual heuristic tuning.

Figure 2: Training and validation loss curves for ZS-SSL. The validation loss (right) exhibits an L-curve behavior, providing a clear signal for early stopping to prevent overfitting.

Performance Highlights

  • Performance Parity: In conditions where training and test data match, ZS-SSL performs nearly as well as supervised models. For Cor-PD knee scans, ZS-SSL achieved a PSNR of 39.55 dB compared to 39.97 dB for supervised learning.
  • Superior Generalization: When faced with a domain shift (e.g., testing on uniform masks after training on random masks), supervised models often fail. ZS-SSL maintained high-quality, artifact-free reconstructions.
  • Efficiency via Transfer Learning: While pure ZS-SSL took approximately 640 seconds per slice, ZS-SSL-TL reduced this to 85 seconds, moving the technique closer to clinical viability.
  • Qualitative Excellence: ZS-SSL successfully removed noise and artifacts in cross-domain scenarios (e.g., using a brain-trained model to reconstruct a knee) where other state-of-the-art methods failed.

Limitations and Future Work

Despite its advantages, ZS-SSL carries higher computational overhead than pre-trained models. Even at 85 seconds per slice with transfer learning, it is slower than millisecond-scale supervised inference. Future work may involve more compact architectures to reduce this time. Additionally, while the stopping criterion is automated, other hyperparameters like learning rates may still require optimization. Extending the framework to 3D volumes or dynamic (4D) MRI would increase available data for self-supervision but also increase the computational burden.

Summary updated

Abstract

Deep learning (DL) has emerged as a powerful tool for accelerated MRI reconstruction, but often necessitates a database of fully-sampled measurements for training. Recent self-supervised and unsupervised learning approaches enable training without fully-sampled data. However, a database of undersampled measurements may not be available in many scenarios, especially for scans involving contrast or translational acquisitions in development. Moreover, recent studies show that database-trained models may not generalize well when the unseen measurements differ in terms of sampling pattern, acceleration rate, SNR, image contrast, and anatomy. Such challenges necessitate a new methodology to enable subject-specific DL MRI reconstruction without external training datasets, since it is clinically imperative to provide high-quality reconstructions that can be used to identify lesions/disease for \emph{every individual}. In this work, we propose a zero-shot self-supervised learning approach to perform subject-specific accelerated DL MRI reconstruction to tackle these issues. The proposed approach partitions the available measurements from a single scan into three disjoint sets. Two of these sets are used to enforce data consistency and define loss during training for self-supervision, while the last set serves to self-validate, establishing an early stopping criterion. In the presence of models pre-trained on a database with different image characteristics, we show that the proposed approach can be combined with transfer learning for faster convergence time and reduced computational complexity. The code is available at \url{https://github.com/byaman14/ZS-SSL}.
BibTeX
@inproceedings{yaman2022zero,
  author = {Burhaneddin Yaman and Seyed Amir Hossein Hosseini and Mehmet Akçakaya},
  title = {{Zero-Shot Self-Supervised Learning for MRI Reconstruction}},
  booktitle = {International Conference on Learning Representations},
  year = {2022},
  url = {https://openreview.net/forum?id=085y6YPaYjP},
}
Figure