DSFNet: Dual-GCN and Location-fused Self-attention with Weighted Fast Normalized Fusion for Polyps Segmentation

Juntong Fan, Tieyong Zeng, Dayang Wang, Wang, DayangarXiv 2023 · DOI · arXiv

Abstract

Polyps segmentation poses a significant challenge in medical imaging due to the flat surface of polyps and their texture similarity to surrounding tissues. This similarity gives rise to difficulties in establishing a clear boundary between polyps and the surrounding mucosa, leading to complications such as local overexposure and the presence of bright spot reflections in imaging. To counter this problem, we propose a new dual graph convolution network (Dual-GCN) and location self-attention mechanisms with weighted fast normalization fusion model, named DSFNet. First, we introduce a feature enhancement block module based on Dual-GCN module to enhance local spatial and structural information extraction with fine granularity. Second, we introduce a location fused self-attention module to enhance the model's awareness and capacity to capture global information. Finally, the weighted fast normalized fusion method with trainable weights is introduced to efficiently integrate the feature maps from encoder, bottleneck, and decoder, thus promoting information transmission and facilitating the semantic consistency. Experimental results show that the proposed model surpasses other state-of-the-art models in gold standard indicators, such as Dice, MAE, and IoU. Both quantitative and qualitative analysis indicate that the proposed model demonstrates exceptional capability in polyps segmentation and has great potential clinical significance.

Introduction

Colorectal cancer often begins as polyps—small growths on the intestinal mucosa. Early detection and accurate segmentation via endoscopy are life-saving, yet polyps are notoriously difficult for AI to "see" due to the "camouflage" effect. The primary challenges include texture similarity, where polyps look almost identical to surrounding healthy tissue, and flat surfaces that lack clear boundaries. Furthermore, endoscopic images often suffer from local overexposure and bright reflections. Existing models like U-Net often struggle with these subtle features, either missing small polyps or producing noisy masks with distorted boundaries.

DSFNet is a novel U-shaped deep learning architecture designed to tackle these issues by combining Dual Graph Convolutional Networks (GCNs) with location-aware self-attention. By leveraging structural graph relationships and a weighted fusion mechanism, it achieves state-of-the-art accuracy on standard medical benchmarks.

Methodology

The DSFNet architecture introduces three major innovations into a U-shaped encoder-decoder framework to improve feature extraction and consistency.

The overall architecture of DSFNet, featuring ConvNext stages, the Dual-GCN based feature enhancement block, and Location-Fused Self-Attention (LFSA) modules.

1. ConvNext Backbone

The encoder utilizes ConvNext blocks instead of standard CNNs. These blocks use larger kernels (7x7) to expand the receptive field, allowing the model to capture more global context from the initial stages of processing.

2. Dual-GCN Feature Enhancement (DBFEB)

Located at the bottleneck, this module treats pixels and regions as nodes in a graph. It utilizes two types of graphs:

  • Spatial Graphs: To capture fine-grained pixel-level relationships.
  • Structural Graphs: Using a novel shortest-path graph attention mechanism to model long-range dependencies across different anatomical regions.

3. Location Fused Self-Attention (LFSA)

In the decoder, standard self-attention is modified by adding trainable row and column offsets. This "location awareness" helps the model understand the relative distance between features, which is crucial for delineating the boundaries of small targets.

4. Weighted Fast Normalized Fusion (FNF)

Rather than simple concatenation, DSFNet uses trainable weights to learn the importance of each feature map during fusion. This ensures semantic consistency across the network by prioritizing the most relevant information from the encoder, bottleneck, and decoder.

Experimental Results

The model was evaluated on two gold-standard polyp segmentation datasets: Endoscene (912 images) and Kvasir-SEG (1,000 images).

Quantitative Performance

DSFNet outperformed several state-of-the-art models, including U-Net, U-Net++, PraNet, and DCRNet. On the Endoscene dataset, DSFNet achieved a Dice coefficient of 87.83% and an IoU of 81.65%. On Kvasir-SEG, it reached a Dice of 90.37% and an IoU of 84.46%.

Qualitative Performance

Visual inspections demonstrate that DSFNet produces cleaner, more faithful boundaries compared to existing methods. While other models often "oversmooth" or create artifacts in overexposed regions, DSFNet maintains structural integrity.

Qualitative comparison of polyp segmentation results across different models. DSFNet shows superior boundary definition and reduced noise compared to U-Net, PraNet, and others.

Conclusion

DSFNet demonstrates exceptional capability in polyp segmentation by effectively addressing the challenges of texture similarity and boundary ambiguity. By integrating graph-based structural modeling with location-aware attention and intelligent feature fusion, the model provides a robust solution for clinical endoscopic assistance. Future work will focus on optimizing computational efficiency for real-time applications and exploring the architecture's generalizability to other medical imaging modalities.

Summary updated

Abstract

Polyps segmentation poses a significant challenge in medical imaging due to the flat surface of polyps and their texture similarity to surrounding tissues. This similarity gives rise to difficulties in establishing a clear boundary between polyps and the surrounding mucosa, leading to complications such as local overexposure and the presence of bright spot reflections in imaging. To counter this problem, we propose a new dual graph convolution network (Dual-GCN) and location self-attention mechanisms with weighted fast normalization fusion model, named DSFNet. First, we introduce a feature enhancement block module based on Dual-GCN module to enhance local spatial and structural information extraction with fine granularity. Second, we introduce a location fused self-attention module to enhance the model's awareness and capacity to capture global information. Finally, the weighted fast normalized fusion method with trainable weights is introduced to efficiently integrate the feature maps from encoder, bottleneck, and decoder, thus promoting information transmission and facilitating the semantic consistency. Experimental results show that the proposed model surpasses other state-of-the-art models in gold standard indicators, such as Dice, MAE, and IoU. Both quantitative and qualitative analysis indicate that the proposed model demonstrates exceptional capability in polyps segmentation and has great potential clinical significance. We have shared our code on anonymous website for evaluation.
BibTeX
@misc{fan2023dsfnet,
  author = {Juntong Fan and Tieyong Zeng and Dayang Wang and Wang, Dayang},
  title = {{DSFNet: Dual-GCN and Location-fused Self-attention with Weighted Fast Normalized Fusion for Polyps Segmentation}},
  year = {2023},
  eprint = {2308.07946},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2308.07946},
}
Figure