High-Resolution 3D MRI With Deep Generative Networks via Novel Slice-Profile Transformation Super-Resolution

Jiahao Lin, Qi Miao, Chuthaporn Surawech, Steven S. Raman, Kai Zhao, Holden H. Wu, Kyunghyun SungIEEE Access 2023 · DOI

TL;DR

This paper introduces a novel deep learning framework, SPTSR, for generating high-resolution 3D MRI images from standard 2D MRI scans. By carefully accounting for the physical characteristics of MRI slice profiles during training and inference, SPTSR significantly improves image quality and reduces artifacts compared to existing methods, paving the way for faster and more detailed MRI scans.

What Problem Does This Paper Address?

Magnetic Resonance Imaging (MRI) is a powerful diagnostic tool, but acquiring high-resolution 3D images is often time-consuming. Standard 2D MRI sequences, like Turbo Spin Echo (TSE), offer good in-plane resolution but suffer from poor through-plane resolution, leading to "staircase artifacts" when reconstructing 3D views. To overcome this, researchers often acquire multiple 2D scans in different orientations, drastically increasing scan time and patient discomfort. While deep learning has shown promise in super-resolution (SR), applying it to the through-plane dimension of 2D MRI is challenging due to the imperfect and overlapping nature of MRI slice profiles. This paper aims to bridge this gap by developing a deep learning approach that can generate high-resolution 3D MRI from a single stack of 2D slices, specifically addressing the limitations imposed by slice profiles.

Methods

The core of this work is the Slice-Profile Transformation Super-Resolution (SPTSR) framework. SPTSR introduces several key innovations:

  • Slice-Profile Downsampling (SP-DS) for Training: Instead of simple downsampling, SPTSR synthesizes low-resolution (LR) training data by convolving high-resolution (HR) images with a realistic model of the slice profile. This ensures the training data accurately reflects the physical characteristics of the MRI acquisition.
  • Slice-Profile Convolution (SP-conv) for Inference: For inference, the input 2D axial scan is pre-processed by convolving it with the slice profile. This aligns the input data with the characteristics of the training data, bridging the physical mismatch between training and testing.
  • Slice-Profile Deconvolution (SP-deconv) for Isotropic SR: After the deep generative network produces a through-plane super-resolved volume, a final SP-deconvolution step is applied. This process removes the blurring artifacts introduced by the slice profile, resulting in truly isotropic high-resolution 3D images.
  • Deep Generative Network Architecture: The authors adapted a Generative Adversarial Network (GAN) architecture (specifically WGAN-GP) for their deep generative model. Key modifications include using a 3-slice input to capture inter-slice relationships and employing 1D anisotropic upsampling to handle the different resolutions in different directions.

Datasets

  • Clinical Prostate MRI Dataset: A large dataset of 4,878 pairs of axial and coronal multi-slice 2D TSE MRI scans was used for training, validation, and testing.
  • Simulated 3D MRI Dataset: For 3D simulation experiments, a dataset of 3D T2w MRI scans was used to generate ground-truth data.

Key Insights and Ideas

  • Slice Profiles Matter: The most crucial insight is that accurately modeling and accounting for MRI slice profiles is essential for effective through-plane super-resolution. Ignoring these profiles leads to artifacts and reduced image quality.
  • Bridging the Training-Testing Gap: SPTSR's success lies in its ability to bridge the physical mismatch between how training data is synthesized (using SP-DS) and how inference data is pre-processed (using SP-conv). This ensures consistency and improves performance.
  • SP-Deconvolution is Key for Isotropy: While the deep generative network improves through-plane resolution, the final SP-deconvolution step is critical for achieving truly isotropic voxels and removing residual blurring.
  • Model Agnosticism: The SPTSR framework is designed to be compatible with various deep learning architectures, suggesting its broad applicability.

Key Results

  • Visual Quality: Expert radiologists rated SPTSR significantly higher in overall image quality, sharpness, and noise reduction compared to baseline methods (bilinear interpolation, KS-ZF, and SMORE). SPTSR achieved near-perfect scores in sharpness and overall quality.
  • Quantitative Analysis (FID): Fréchet Inception Distance (FID) scores, a measure of image perceptual quality, confirmed SPTSR's superiority, indicating it generates images most similar to the ground truth.
  • Through-plane SR: SPTSR achieved a 5.5x improvement in through-plane resolution for clinical 2D MRI scans.
  • Isotropic SR (3D Simulation): In 3D simulation experiments, SPTSR achieved 4x isotropic super-resolution, with results closely matching the 3D ground truth in all views, outperforming SMORE.
  • Efficiency: The method allows for high-resolution 3D reconstruction from a single 2D scan, potentially reducing overall MRI scan time.

Limitations and Future Work

  • Inter-slice Motion Artifacts: The current approach does not explicitly account for inter-slice motion artifacts, which can be significant in certain applications like fetal brain MRI.
  • Through-plane SR Loss Function: The current network training primarily focuses on through-plane SR loss. Future work could incorporate self-consistency losses in the in-plane orientation to further improve isotropic SR.
  • Motion Robustness: The authors suggest that a more motion-robust SR framework could be developed by including simulated motion in the training data or enforcing regularization terms.
  • Deep Learning Advancements: The SPTSR framework's model-agnostic nature allows for the integration of future advancements in deep learning architectures and training schemes.

Summary updated

BibTeX
@article{lin2023high,
  author = {Jiahao Lin and Qi Miao and Chuthaporn Surawech and Steven S. Raman and Kai Zhao and Holden H. Wu and Kyunghyun Sung},
  title = {{High-Resolution 3D MRI With Deep Generative Networks via Novel Slice-Profile Transformation Super-Resolution}},
  journal = {IEEE Access},
  year = {2023},
  doi = {10.1109/access.2023.3307577},
}
Figure