A Generative Adversarial Network Technique for High-Quality Super-Resolution Reconstruction of Cardiac Magnetic Resonance Images

Ming Zhao, Yang Wei, Kelvin K. L. WongResearch Square 2021 · DOI

TL;DR

Researchers have developed DnSRGAN, a novel deep learning framework that combines denoising and super-resolution to turn grainy, low-resolution cardiac MRIs into high-fidelity images. By integrating a pre-denoising step and stabilizing training with Wasserstein GAN techniques, the model outperforms standard methods, providing clearer details for better clinical diagnosis.

What Problem Does This Paper Address?

Cardiac Magnetic Resonance (CMR) imaging is a gold standard for diagnosing heart disease, but it’s notoriously difficult to get "perfect" shots. Patient movement, irregular heartbeats (arrhythmia), and breathing issues often introduce noise and artifacts. Furthermore, high-resolution scans take a long time to acquire, which is taxing for patients.

While Super-Resolution (SR) — the process of computationally upscaling low-resolution (LR) images — seems like a natural solution, it has a major "garbage in, garbage out" problem. If the input image is noisy, standard SR algorithms (like SRGAN or ESRGAN) often mistake that noise for actual tissue texture, amplifying artifacts and creating "hallucinated" features that could lead to a misdiagnosis.

Methods

The authors propose a two-stage architecture called DnSRGAN (Denoising Super-resolution Generative Adversarial Network):

  1. The Denoiser (DnCNN): Before upscaling, the image passes through a feed-forward denoising convolutional neural network. This ensures the Generative Adversarial Network (GAN) receives a "clean" signal, preventing it from wasting capacity on reconstructing noise.
  2. The Generator: Based on the SRGAN framework, this component learns the mapping between LR and HR (High-Resolution) images, specifically aiming for a 4x magnification.
  3. The Stabilizer (WGAN-GP): Standard GANs are notoriously unstable and prone to "gradient vanishing," where the model stops learning. The authors implemented Wasserstein GAN with Gradient Penalty (WGAN-GP). This enforces a "Lipschitz constraint," ensuring the training process is smoother and more mathematically stable.
  4. Hybrid Loss Function: Instead of just looking at pixel-to-pixel differences (MSE), the model uses a weighted combination of MSE, VGG (perceptual loss), and WGAN adversarial loss. This helps the model prioritize "looking right" to a human eye rather than just matching raw pixel values.

Datasets

The study utilized a dataset of cardiac images from 64 patients collected using a Siemens Sonata 1.5 T scanner.

  • Total Images: 675 MR images.
  • Training Set: 600 images, which were further "amplified" using data augmentation techniques like cropping, translation, and rotation to improve the model's robustness.
  • Test Set: 75 images, divided into three groups of 25 to verify the model’s consistency across different time-series frames.

Key Insights and Ideas

  • Pre-processing is Half the Battle: The most significant "aha moment" is that GAN performance drops sharply when input PSNR (Peak Signal-to-Noise Ratio) is low. By "cleaning" the image first with DnCNN, the GAN can focus entirely on structural reconstruction.
  • Stability Equals Speed: By switching to a WGAN-GP approach, the researchers reduced the required training epochs from 10,000 down to just 1,500. The model reached convergence much faster and produced more reliable results without the "unreal texture details" common in traditional GANs.
  • Perceptual over Pixel-Perfect: The paper reinforces that for medical imaging, matching the features (via VGG loss) is often more important than matching the pixels (via MSE), as it preserves the sharp boundaries of heart valves and chambers.

Key Results

The DnSRGAN was compared against three industry standards: Bicubic interpolation, SRGAN, and ESRGAN.

  • Quantitative Superiority: DnSRGAN achieved significantly higher PSNR and SSIM (Structural Similarity Index) scores across all test groups. For instance, the Avg-PSNR was visibly higher in the DnSRGAN results compared to the 25–30 dB range of other methods.
  • Qualitative Clarity: Visual comparisons (see Figure 2 in the paper) show that while Bicubic methods are blurry and standard SRGANs often look "mottled," DnSRGAN produces images that are remarkably close to the Ground Truth (GT), with sharp edges and minimal graininess.

Limitations and Future Work

Despite the impressive results, the authors acknowledge a few hurdles:

  • Data Diversity: The current dataset consists of cine MR images (time-series of the same object). While augmentation helped, the variety of cardiac pathologies was limited.
  • Dimensionality: The current work focuses on 2D slices. The authors suggest that 3D super-resolution—incorporating spatial features across the entire volume of the heart—is the next logical step.
  • Synthetic Data: They are exploring using deep learning to generate entirely new, synthetic CMR images to further expand training sets without needing more patient scans.

Ultimately, DnSRGAN represents a promising step toward automated, high-quality medical imaging that could help doctors catch heart conditions earlier and with greater confidence.

Summary updated

Abstract

<title>Abstract</title> Objective: In this paper, we proposed a Denoising Super-resolution Generative Adversarial Network (DnSRGAN) method for high-quality super-resolution reconstruction of noisy cardiac magnetic resonance (CMR) images.Methods: This method is based on feed-forward denoising convolutional neural network (DnCNN) and SRGAN architecture. Firstly, we used a feed-forward denoising neural network to pre-denoise the CMR image to ensure that the input is a clean image. Secondly, we use the gradient penalty (GP) method to solve the problem of the discriminator gradient disappearing, which improves the convergence speed of the model. Finally, a new loss function is added to the original SRGAN loss function to monitor GAN gradient descent to achieve more stable and efficient model training, thereby providing higher perceptual quality for the super-resolution of CMR images.Results: We divided the tested cardiac images into 3 groups, each group of 25 images, calculated the Peak Signal to Noise Ratio (PSNR) /Structural Similarity (SSIM) between Ground Truth (GT) and the images generated by super-resolution, used them to evaluate our model, and Compared with the current widely used method: Bicubic ESRGAN and SRGAN, our method has better reconstruction quality and higher PSNR/SSIM score.Conclusion: We used DnCNN to denoise the CMR image, and then using the improved SRGAN to perform super-resolution reconstruction of the denoised image, we can solve the problem of high noise and artifacts that cause the cardiac image to be reconstructed incorrectly during super-resolution. Furthermore, our method is capable of high-quality reconstruction of noisy cardiac images.
BibTeX
@inproceedings{zhao2021generative,
  author = {Ming Zhao and Yang Wei and Kelvin K. L. Wong},
  title = {{A Generative Adversarial Network Technique for High-Quality Super-Resolution Reconstruction of Cardiac Magnetic Resonance Images}},
  booktitle = {Research Square},
  year = {2021},
  doi = {10.21203/rs.3.rs-196187/v1},
}
Figure