DSGA-Net: Deeply separable gated transformer and attention strategy for medical image segmentation network

Chang-Tien Lu, Ieee Fellow, Junding Sun, Jiuqiang Zhao, Xiaosheng Wu, Chaosheng Tang, Shuihua Wang, Yudong ZhangJ. King Saud Univ. Comput. Inf. sci 2023 · DOI

TL;DR

This paper introduces DSGA-Net, a novel deep learning architecture for medical image segmentation that combines a "Deeply Separable Gated Visual Transformer" (DSG-ViT) with a "Mixed Three-branch Attention" (MTA) module. DSGA-Net aims to improve the segmentation of small and complex organs by enhancing both local and global feature extraction and reducing information loss during the upsampling process, achieving state-of-the-art results on several benchmark datasets.

What Problem Does This Paper Address?

Accurate segmentation of organs and lesions in medical images is crucial for clinical diagnosis and research. However, traditional methods often struggle with:

  • Under-segmentation and over-segmentation: Especially for small or irregularly shaped organs, leading to inaccurate boundaries.
  • Information loss during upsampling: As images are processed through deep networks, fine details can be lost, making it difficult to reconstruct precise segmentation masks.
  • Limited contextual understanding: Convolutional Neural Networks (CNNs) have a limited receptive field, making it challenging to capture long-range dependencies and global context.
  • Data scarcity: Medical imaging datasets can be relatively small, posing a challenge for training complex models.

While Vision Transformers (ViTs) have shown promise in capturing global context, they can be computationally expensive and may not always excel at precise localization, especially with limited data. This paper seeks to bridge these gaps by integrating the strengths of transformers and attention mechanisms in a novel architecture.

Methods

DSGA-Net is built upon two key novel components:

  1. Deeply Separable Gated Visual Transformer (DSG-ViT) Block: This module enhances the encoder by incorporating a gated self-attention mechanism. It aims to improve:
  2. Local and global feature interaction: By processing features within windows and between windows.
  3. Channel-wise feature understanding: Enhancing sensitivity to location information.
  4. Gated position embeddings: This is a crucial addition that allows the model to control the influence of positional information, making it more robust to variations and improving localization accuracy, especially when dealing with limited data.
  5. Mixed Three-branch Attention (MTA) Module: This module is strategically placed between the encoder and decoder layers. Its purpose is to:
  6. Fuse multi-dimensional features: It combines channel attention, spatial attention, and global self-attention to comprehensively capture feature information.
  7. Compensate for information loss: By effectively extracting and re-introducing rich semantic and accurate location information during the upsampling process, it helps recover details lost during downsampling.

The overall DSGA-Net architecture follows a U-Net-like structure, with a four-layer DSG-ViT encoder and a cascaded decoder, interspersed with MTA modules for feature fusion.

Datasets

The paper evaluates DSGA-Net on three publicly available medical image segmentation datasets:

  • Synapse Multi-Organ Segmentation Dataset: This dataset consists of 30 abdominal CT scans with 3779 axial slices, used for segmenting eight abdominal organs.
  • Automated Cardiac Diagnostic Challenge (ACDC) Dataset: This dataset comprises MRI images for segmenting cardiac structures (left ventricle, right ventricle, and myocardium).
  • Brain Tumor Segmentation Challenge 2020 (BraTs2020) Dataset: This dataset includes multi-modal MRI scans for brain tumor segmentation, classifying different tumor regions (background, NCR/NET, ED, ET).

Key Insights and Ideas

  • Gated Attention for Localization: The introduction of a gating mechanism within the self-attention of DSG-ViT is a key innovation. It allows the model to dynamically control the impact of positional information, which is critical for accurate segmentation, especially in data-scarce medical imaging scenarios.
  • Multi-Branch Attention for Feature Fusion: The MTA module's strategy of combining channel, spatial, and global attention is a robust way to ensure that rich, multi-faceted feature information is passed from the encoder to the decoder, mitigating information loss during upsampling.
  • Transformer-CNN Synergy: The paper effectively demonstrates how combining transformer-based feature extraction (for global context) with attention mechanisms and a U-Net-like decoder (for local details and reconstruction) can lead to superior segmentation performance.
  • Robustness to Resolution: DSGA-Net shows improved robustness to input image resolution compared to baseline models, suggesting its adaptability to different imaging protocols.

Key Results

DSGA-Net achieved state-of-the-art results across the evaluated datasets:

  • Synapse Dataset: Achieved a Dice Similarity Coefficient (DSC) of 81.24%, outperforming the baseline TransUNet by 3.76% and reducing the Hausdorff Score (HD) to 20.91%.
  • ACDC Dataset: Showed significant improvements in average DSC, outperforming TransUNet by 1.63%.
  • BraTs2020 Dataset: Achieved an average DSC of 85.82%, with a notable reduction in HD to 5.27% compared to TransUNet.

Qualitative results (visualizations) also demonstrated DSGA-Net's superior ability to delineate organ boundaries accurately, especially for smaller organs, and reduce over/under-segmentation compared to other methods.

Limitations and Future Work

The authors acknowledge the following limitations:

  • Computational Burden: Transformer-based models, including DSGA-Net, can be computationally intensive, requiring significant GPU memory and potentially limiting their usability on less powerful hardware.
  • Parameter Count: While not explicitly stated as a limitation, the complexity of the model suggests a higher parameter count, which can be a concern for deployment.

Future work suggested by the authors includes:

  • Reducing Computational Complexity: Developing methods to decrease the computational cost and memory footprint of the model.
  • Optimizing Parameter Count: Exploring ways to reduce the number of parameters without sacrificing performance.
  • Increasing Inference Speed: Focusing on faster prediction times for real-world applications.
  • Applying to Other Segmentation Tasks: Extending the model's applicability to a wider range of medical imaging segmentation challenges.

Summary updated

BibTeX
@article{fellow2023dsga,
  author = {Chang-Tien Lu, IEEE Fellow and Junding Sun and Jiuqiang Zhao and Xiaosheng Wu and Chaosheng Tang and Shuihua Wang and Yudong Zhang},
  title = {{DSGA-Net: Deeply separable gated transformer and attention strategy for medical image segmentation network}},
  journal = {J. King Saud Univ. Comput. Inf. sci},
  year = {2023},
  doi = {10.1016/j.jksuci.2023.04.006},
}
Figure