Generalising Deep Learning MRI Reconstruction across Different Domains

Cheng Ouyang, Jo Schlemper, Carlo Biffi, Gavin Seegoolam, Jose Caballero, Anthony N. Price, Joseph V. Hajnal, Daniel Rueckert

Abstract

We look into robustness of deep learning based MRI reconstruction when tested on unseen contrasts and organs. We then propose to generalise the network by training with large publicly-available natural image datasets with synthesised phase information to achieve high cross-domain reconstruction performance which is competitive with domain-specific training. To explain its generalisation mechanism, we have also analysed patch sets for different training datasets.

Introduction

Deep learning has revolutionized MRI reconstruction by enabling high-quality image recovery from undersampled data. However, these models often suffer from a significant "domain shift" problem: a network trained on a specific organ or contrast type typically performs poorly when applied to different anatomical regions. The standard solution—training specialized models for every clinical scenario—is often impractical due to the high cost and difficulty of obtaining fully-sampled ground truth MRI data. This paper explores a novel approach: building a robust "generalist" model by training on massive datasets of natural images.

Methodology

The proposed method utilizes a deep-cascade of Convolutional Neural Networks (CNNs) to learn image priors. The core of the strategy involves training on the MS-COCO dataset, which contains over 118,000 natural images. To bridge the gap between magnitude-only natural images and complex-valued MRI data, the authors introduce synthetic phase information during training. This allows the model to learn physics-based constraints applicable to MRI. Additionally, the researchers performed a patch analysis (using 7x7 patches) to quantify the visual similarity between natural images and various MRI domains using Euclidean nearest neighbor distances.

Results and Discussion

The model trained on MS-COCO demonstrated exceptional robustness, achieving high-fidelity reconstructions across diverse targets including Cardiac CINE, Knee-CPD (proton-density weighted), and Knee-AT2 (T2 weighted) scans.

Visual comparison of reconstruction quality and error maps across different source domains for a cardiac target. The MS-COCO trained model (Proposed) shows reconstruction quality comparable to domain-specific training.

Quantitative Performance

Quantitative evaluation confirms that the MS-COCO "generalist" model is highly competitive with domain-specific "specialists." As shown in the performance table, the MS-COCO model often outperforms models trained on different MRI domains and even matches those trained on the target domain itself. For instance, on Knee-CPD data, the MS-COCO model achieved a PSNR of 33.79, surpassing the performance of the model trained specifically on that domain (33.42).

Quantitative performance (PSNR and SSIM) of models trained on different source domains and tested across various MRI datasets. Bold values indicate the best performance for each test set.

Generalisation Mechanism

Patch analysis provided a mathematical explanation for this success: MS-COCO patches were found to be "closer" to all MRI domains than the MRI domains were to each other. This suggests that the diversity of natural images acts as a universal feature set that encompasses the local structures found in various medical scans. The inclusion of synthetic phase is essential for this cross-domain feature sharing.

Conclusion

This study demonstrates that large-scale natural image datasets can effectively substitute for scarce medical data in training robust MRI reconstruction models. By incorporating synthetic phase, a generalist model can achieve performance parity with specialized models across multiple anatomical regions. Future work will focus on extending this framework to 3D and multi-coil imaging, as well as exploring transfer learning for further refinement.

Summary updated

Abstract

We look into robustness of deep learning based MRI reconstruction when tested on unseen contrasts and organs. We then propose to generalise the network by training with large publicly-available natural image datasets with synthesised phase information to achieve high cross-domain reconstruction performance which is competitive with domain-specific training. To explain its generalisation mechanism, we have also analysed patch sets for different training datasets.
BibTeX
@misc{ouyang2019generalising,
  author = {Cheng Ouyang and Jo Schlemper and Carlo Biffi and Gavin Seegoolam and Jose Caballero and Anthony N. Price and Joseph V. Hajnal and Daniel Rueckert},
  title = {{Generalising Deep Learning MRI Reconstruction across Different Domains}},
  year = {2019},
  eprint = {1902.10815},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1902.10815},
}
Figure