Feedback Graph Attention Convolutional Network for Medical Image Enhancement

Xiaobin Hu, Yanyang Yan, Wenqi Ren, Hongwei Li, Yu Zhao, Amirhossein Bayat, Bjoern H. MenzearXiv 2020 · OpenReview

TL;DR

This paper introduces the Feedback Graph Attention Convolutional Network (FB-GACN), a novel deep learning architecture for enhancing medical MRI images. By treating image sub-regions as nodes in a graph and leveraging attention mechanisms, FB-GACN effectively captures global structural similarities to improve image quality, remove artifacts, and boost resolution, outperforming existing methods.

What Problem Does This Paper Address?

Magnetic Resonance Imaging (MRI) is a powerful diagnostic tool, but it faces inherent trade-offs between image resolution, signal-to-noise ratio (SNR), and acquisition time. To achieve higher resolution and better detail, longer scan times are often required, which can lead to motion artifacts and reduced SNR. Furthermore, different MRI acquisition protocols can result in varying image qualities, making it challenging to compare or combine data. This paper tackles the problem of enhancing degraded MRI images by addressing both super-resolution (increasing image resolution) and artifact removal, aiming to produce clearer, more informative images without compromising acquisition efficiency.

Methods

The core innovation of FB-GACN lies in its unique architecture, which combines several key components:

  • Graph Similarity Branch: This branch constructs a graph where each image sub-region is treated as a node. These nodes are then linked based on their similarity, allowing the network to exploit global structural information and symmetry across different parts of the image. This is achieved using Graph Attention Network (GAT) layers, which learn to assign weights to neighboring nodes based on their relevance.
  • Content Branch: This branch focuses on extracting and enhancing texture details within the image, working in parallel with the graph similarity branch.
  • Feedback Mechanism: A recurrent structure is employed to refine low-level image representations by iteratively incorporating high-level information from both the graph similarity and content branches. This feedback loop allows for progressive improvement of image quality and texture details.
  • Reconstruction: Finally, an upsampling block reconstructs the enhanced, super-resolved image by utilizing the learned correlations and texture priors from the preceding branches.

The network is trained using an L1 norm-based loss function, aiming to minimize the difference between the reconstructed and ground truth images.

Datasets

The authors evaluated their method on two distinct tasks using the following datasets:

  • Cross-Protocol Super-Resolution (Diffusion MRI): The MUSHAC dataset was used. High-resolution (HR) images were acquired with a Prisma scanner (1.5 × 1.5 × 1.5 mm³ voxel size), and low-resolution (LR) images were acquired with a larger voxel size (2.4 × 2.4 × 2.4 mm³). Nine subjects were used for training, and one for testing.
  • Artifact Removal (FLAIR MRI): The WMH dataset was utilized. The dataset was randomly split into training (2225 images from 48 patients), validation (278 images from 6 patients), and test sets (278 images from 6 patients). Simulated artifacts, mimicking MRI motion artifacts, were generated for the FLAIR modality.

Key Insights and Ideas

  • Graph Representation for Global Structure: The idea of representing image sub-regions as nodes in a graph to capture global structural relationships is a powerful way to move beyond purely local feature extraction, which is common in many CNNs.
  • Self-Similarity as a Priori: Exploiting the inherent self-similarity and symmetry within medical images, particularly in MRI, provides valuable prior information that can significantly aid in reconstruction and artifact removal.
  • Iterative Refinement via Feedback: The feedback mechanism, allowing for multiple iterations of refinement, is crucial for progressively enhancing image quality. This suggests that a single pass might not be sufficient for complex image restoration tasks.
  • Synergy of Graph and Content: The parallel processing of graph similarity and content branches allows the model to leverage both global structural context and local texture details, leading to more comprehensive enhancement.

Key Results

The FB-GACN model demonstrated superior performance on both experimental tasks:

  • Cross-Protocol Super-Resolution: The proposed method achieved better results compared to state-of-the-art methods, notably showing a 3.46 dB improvement over the traditional bicubic interpolation method.
  • Artifact Removal: For artifact removal and super-resolution at a 2x scale, FB-GACN outperformed existing algorithms, achieving the best PSNR of 25.78 dB and SSIM of 0.7839.
  • Ablation Studies: Removing the graph similarity branch led to a slight decrease in performance for both tasks, highlighting the importance of the graph-based approach. The iterative number of the feedback connection was explored, with T=4 iterations showing the best balance between performance and computational cost.

Qualitative results (visual comparisons in Figures 3 and 4) further supported these findings, showing that the FB-GACN produced images with higher visual quality, clearer structures, finer contrast, and reduced artifacts.

Limitations and Future Work

  • Computational Cost: While the feedback mechanism improves performance, it also increases computational complexity. The authors chose T=4 iterations as a balance, suggesting that further optimization for speed might be beneficial.
  • Generalizability: The paper focuses on specific MRI modalities (diffusion MRI and FLAIR). Future work could explore the applicability of FB-GACN to other MRI sequences or even different medical imaging modalities.
  • Dataset Size: While the datasets used are standard, the performance of deep learning models is often highly dependent on the size and diversity of training data. Exploring larger, more varied datasets could further enhance robustness.
  • Hyperparameter Tuning: The paper mentions using ADAM optimizer and a specific learning rate schedule. Further investigation into optimal hyperparameter settings for different scenarios could be valuable.

Summary updated

Abstract

Artifacts, blur and noise are the common distortions degrading MRI images during the acquisition process, and deep neural networks have been demonstrated to help in improving image quality. To well exploit global structural information and texture details, we propose a novel biomedical image enhancement network, named Feedback Graph Attention Convolutional Network (FB-GACN). As a key innovation, we consider the global structure of an image by building a graph network from image sub-regions that we consider to be node features, linking them non-locally according to their similarity. The proposed model consists of three main parts: 1) The parallel graph similarity branch and content branch, where the graph similarity branch aims at exploiting the similarity and symmetry across different image sub-regions in low-resolution feature space and provides additional priors for the content branch to enhance texture details. 2) A feedback mechanism with a recurrent structure to refine low-level representations with high-level information and generate powerful high-level texture details by handling the feedback connections. 3) A reconstruction to remove the artifacts and recover super-resolution images by using the estimated sub-region correlation priors obtained from the graph similarity branch. We evaluate our method on two image enhancement tasks: i) cross-protocol super resolution of diffusion MRI; ii) artifact removal of FLAIR MR images. Experimental results demonstrate that the proposed algorithm outperforms the state-of-the-art methods.
BibTeX
@misc{hu2020feedback,
  author = {Xiaobin Hu and Yanyang Yan and Wenqi Ren and Hongwei Li and Yu Zhao and Amirhossein Bayat and Bjoern H. Menze},
  title = {{Feedback Graph Attention Convolutional Network for Medical Image Enhancement}},
  year = {2020},
  eprint = {2006.13863},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2006.13863},
}
Figure