Invertible Sharpening Network for MRI Reconstruction Enhancement

Siyuan Dong, Eric Z. Chen, Lin Zhao, Xiao Chen, Yikang Liu, Terrence Chen, Shanhui Sun

TL;DR

Researchers have developed InvSharpNet, an invertible neural network that sharpens blurry MRI reconstructions by flipping the standard training paradigm on its head. By learning to "blur" ground-truth images into reconstructions during training, the network leverages its mathematical invertibility to "un-blur" images during clinical use, providing high-frequency detail without the "hallucinated" artifacts common in GAN-based methods.

What Problem Does This Paper Address?

Magnetic Resonance Imaging (MRI) is notoriously slow. To speed it up, technicians often perform "under-sampling," taking fewer measurements than required. Deep learning models are then used to reconstruct the full image.

However, these models face a fundamental hurdle: the "solution averaging" problem. Because MRI reconstruction is an ill-posed "one-to-many" problem (one set of under-sampled data could correspond to several slightly different sharp images), standard training using Mean Squared Error (MSE) or Mean Absolute Error (MAE) results in the model outputting the pixel-wise average of all possible solutions. This leads to a safe but frustratingly blurry image. While Generative Adversarial Networks (GANs) can create sharper images, they often "hallucinate" fine details that aren't actually there—a dangerous trade-off in medical diagnostics.

Methods

The authors propose InvSharpNet, which utilizes an Invertible Neural Network (INN) architecture based on iResNet. The core innovation is the Backward Training strategy:

  1. The Flip: Instead of training the network to map a blurry image to a sharp one (one-to-many), they train it to map the sharp ground truth to the blurry reconstruction (one-to-one). This is a well-defined, stable mapping.
  2. Invertibility: Because the network is invertible, once it learns how to "blur" a sharp image into a specific reconstruction, it mathematically knows how to reverse the process.
  3. Inference: During deployment, the blurry reconstruction is fed into the network in the forward direction to produce a sharpened output.
  4. Constraints: To ensure the network is truly invertible, the authors enforce a Lipschitz constraint (specifically, the Lipschitz constant must be $< 1$) on the convolutional layers and use fixed-point iterations to compute the inverse.

Datasets

The model was evaluated on three major datasets:

  • fastMRI Knee dataset: 34,742 training images and 7,135 evaluation images.
  • fastMRI Brain dataset: 67,558 training images and 20,684 evaluation images.
  • In-house Brain dataset: 584 images featuring five different gradient echo (GRE) contrasts, used to test the model's ability to generalize to data outside the fastMRI ecosystem.

Key Insights and Ideas

  • One-to-One is Better than One-to-Many: By re-framing the sharpening task as the inverse of a blurring task, the authors bypass the statistical "averaging" that plagues traditional feed-forward networks.
  • Lipschitz Balancing Act: There is a trade-off between a network's learning capacity and its invertibility. A lower Lipschitz constant (e.g., 0.5) makes inversion easy but limits what the network can learn; a higher constant (e.g., 0.9) allows for complex learning but requires more computational iterations to invert. The authors settled on 0.7 as the "sweet spot."
  • Metrics vs. Visuals: The paper highlights that standard metrics like PSNR and SSIM often penalize sharp images because they focus on pixel-wise similarity rather than perceptual sharpness. They argue that the contrast term within SSIM and LPIPS (Learned Perceptual Image Patch Similarity) are better indicators of clinical utility.

Key Results

  • Radiologist Approval: In a blind study, two radiologists rated InvSharpNet (backward) with a 4.67/5.0 for sharpness, significantly higher than the 4.00/5.0 of the baseline reconstruction model.
  • Artifact Reduction: Unlike GANs (cWGAN), which introduced visible artifacts (hallucinated features), InvSharpNet maintained high artifact-resistance scores (4.00/5.0), matching the baseline while being much sharper.
  • Perceptual Gains: InvSharpNet achieved the best (lowest) LPIPS scores across all datasets, indicating its outputs were perceptually closest to the ground truth.
  • Generalizability: The model trained on fastMRI data performed well on the in-house GRE dataset, suggesting the learned sharpening transform is robust across different scanner types and sequences.

Limitations and Future Work

  • Computational Cost: The requirement for fixed-point iterations to compute the inverse increases the memory usage and training time compared to standard CNNs.
  • Network Size: To compensate for the restricted learning capacity caused by Lipschitz constraints, the authors had to use a relatively large model (5.4M parameters). A smaller version (0.7M parameters) showed significantly higher error.
  • Future Directions: The authors suggest this "backward training" invertible approach could be extended beyond MRI to other image restoration tasks like general denoising and super-resolution.

Summary updated

Abstract

High-quality MRI reconstruction plays a critical role in clinical applications. Deep learning-based methods have achieved promising results on MRI reconstruction. However, most state-of-the-art methods were designed to optimize the evaluation metrics commonly used for natural images, such as PSNR and SSIM, whereas the visual quality is not primarily pursued. Compared to the fully-sampled images, the reconstructed images are often blurry, where high-frequency features might not be sharp enough for confident clinical diagnosis. To this end, we propose an invertible sharpening network (InvSharpNet) to improve the visual quality of MRI reconstructions. During training, unlike the traditional methods that learn to map the input data to the ground truth, InvSharpNet adapts a backward training strategy that learns a blurring transform from the ground truth (fully-sampled image) to the input data (blurry reconstruction). During inference, the learned blurring transform can be inverted to a sharpening transform leveraging the network’s invertibility. The experiments on various MRI datasets demonstrate that InvSharpNet can improve reconstruction sharpness with few artifacts. The results were also evaluated by radiologists, indicating better visual quality and diagnostic confidence of our proposed method.
BibTeX
@inproceedings{dong2022invertible,
  author = {Siyuan Dong and Eric Z. Chen and Lin Zhao and Xiao Chen and Yikang Liu and Terrence Chen and Shanhui Sun},
  title = {{Invertible Sharpening Network for MRI Reconstruction Enhancement}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2022},
  url = {https://openreview.net/forum?id=y4Gv8I5L9Y},
}

Loading related content...

Figure