Multi-Attention Segmentation Networks Combined with the Sobel Operator for Medical Images

Fangfang Lu, Chi Tang, Tianxiang Liu, Zhihao Zhang, Leida Li2023 · DOI

TL;DR

This paper introduces SMA-Net, a novel deep learning architecture for segmenting COVID-19 lesions in CT scans. By integrating Sobel edge detection and a dual attention mechanism (channel and spatial), SMA-Net significantly improves lesion segmentation accuracy, particularly for small and subtle lesions, outperforming existing methods.

What Problem Does This Paper Address?

Accurate segmentation of lung lesions in CT scans is crucial for diagnosing and assessing the severity of COVID-19. Manual segmentation is time-consuming and prone to inter-observer variability. While deep learning methods have shown promise, they often struggle with:

  • Loss of fine details: The downsampling process in encoder-decoder networks can discard important edge and texture information of lesions.
  • Insufficient feature learning: Existing models may not effectively capture the complex relationships between channel and spatial features within the images.
  • Difficulty with small lesions: Standard segmentation loss functions can be insensitive to small, subtle lesions, leading to under-segmentation.

This paper aims to address these limitations by proposing a more robust and accurate segmentation network.

Methods

The proposed SMA-Net is an end-to-end segmentation network built upon an encoder-decoder architecture. Its key novelties include:

  • Edge Feature Fusion Module: This module incorporates the Sobel operator to extract edge information from the CT images. This edge information is then fused with the original image to provide richer spatial detail to the network, helping to preserve lesion boundaries.
  • Self-Attentive Channel Attention Mechanism (SCAM): Unlike traditional SE modules that use global average pooling (which can lose semantic information), SCAM uses a self-attentive approach to learn channel interdependencies. It computes attention weights by performing matrix multiplication on reshaped feature maps, allowing it to better emphasize informative channels.
  • Spatial Linear Attention Mechanism (PLAM): To address the computational complexity of standard self-attention mechanisms (which are quadratic in resolution), PLAM employs a linear attention approach. This reduces the complexity to linear, making it more efficient and scalable for high-resolution medical images while still effectively capturing spatial relationships and focusing on key regions.
  • Tversky Loss Function: This loss function is chosen to better handle the class imbalance inherent in medical image segmentation, particularly for small lesions. By adjusting parameters $\alpha$ and $\beta$, it can control the trade-off between false positives and false negatives, improving sensitivity to small lesion areas.

Datasets

The study utilized a public dataset from zendo [41] containing 20 COVID-19 CT scans with lung and lesion segmentation labels. A total of 2237 CT images were selected for the experiment. The dataset was annotated by two radiologists.

Key Insights and Ideas

  • Edge information is critical for detail: Explicitly incorporating edge features using the Sobel operator significantly enhances the network's ability to capture fine details, which are often lost during downsampling.
  • Attention mechanisms are powerful but need efficiency: While attention mechanisms are vital for focusing on relevant features, their computational cost can be prohibitive. The proposed SCAM and PLAM modules offer efficient ways to leverage channel and spatial attention, respectively.
  • Loss function choice matters for imbalanced data: The Tversky loss function proved crucial for improving the segmentation of small, subtle lesions, a common challenge in COVID-19 CT analysis.

Key Results

SMA-Net achieved state-of-the-art performance on the COVID-19 lesion segmentation task.

  • Quantitative: The proposed SMA-Net model achieved an average Dice Similarity Coefficient (DSC) of 86.1% and an Intersection over Union (IOU) of 77.8%, outperforming most existing segmentation networks. Specifically, it showed a 7.8% improvement in IOU compared to UNet.
  • Qualitative: Visual comparisons demonstrated that SMA-Net's segmentation results were closest to the ground truth, with fewer false positives and better delineation of small lesion regions compared to other methods.

Limitations and Future Work

The authors acknowledge the following limitations and suggest future directions:

  • High Computational Complexity: SMA-Net requires significant computational resources, which might be a barrier for deployment in resource-constrained hospital settings.
  • Lack of Classification Capability: SMA-Net is solely a segmentation network and does not perform classification tasks simultaneously.
  • Future Work:
    • Model Lightweighting: Explore techniques like expanded convolutions and pruning to reduce the network's complexity without sacrificing performance.
  • Integrated Classification: Integrate a classification module into the network to achieve simultaneous segmentation and classification.
  • Semi-Supervised Learning: Investigate semi-supervised approaches, similar to Inf-Net, to reduce reliance on fully labeled datasets.

Summary updated

BibTeX
@misc{lu2023multi,
  author = {Fangfang Lu and Chi Tang and Tianxiang Liu and Zhihao Zhang and Leida Li},
  title = {{Multi-Attention Segmentation Networks Combined with the Sobel Operator for Medical Images}},
  year = {2023},
  doi = {10.3390/s23052546},
}
Figure