DGCFNet: Dual Global Context Fusion Network for remote sensing image semantic segmentation

Yuan Liao, Tongchi Zhou, Lu Li, Jinming Li, Jiuhao Shen, Askar HamdullaPeerJ Comput. Sci 2025 · DOI

TL;DR

This paper introduces DGCFNet, a novel deep learning architecture for semantic segmentation of remote sensing images. By effectively fusing local features from CNNs with global contextual information from Transformers, DGCFNet achieves state-of-the-art performance on challenging datasets, improving the accuracy of identifying objects like buildings and water bodies.

What Problem Does This Paper Address?

Remote sensing images present unique challenges for semantic segmentation, including complex backgrounds, similar appearances between different object classes (inter-class similarity), and significant variations within the same object class (intra-class variance). Traditional Convolutional Neural Networks (CNNs) excel at capturing local details but struggle with long-range dependencies. While Transformers are adept at modeling global context, they can sometimes overlook fine-grained local features. This paper aims to bridge this gap by developing a hybrid CNN-Transformer architecture that leverages the strengths of both to achieve more accurate and robust semantic segmentation of remote sensing imagery.

Methods

The proposed DGCFNet employs an encoder-decoder structure. The encoder utilizes a ResNet18 backbone to extract multi-scale features. The decoder is where the novel contributions lie:

  • Dual-Branch Global Extraction Module (DGEM): This module is designed to capture both local and global contextual information. It comprises:
  • Global Attention Branch (GAB): Uses a multi-head self-attention mechanism with cross-window interactions to capture global context.
  • Global Compensation Branch (GCB): Employs multiple average pooling layers of varying sizes to extract multi-scale local contextual information and preserve local details.
  • Cross-Level Information Interaction Module (CIIM): This module, based on Transformer principles, enhances the correlation between features from different levels of the network. It fuses information from adjacent layers to improve contextual understanding.
  • Feature Interaction Guided Module (FIGM): This module adaptively fuses the intra-layer global context from DGEM with the inter-layer context from CIIM, aiming to refine segmentation results and improve boundary smoothness.

Datasets

The proposed DGCFNet was evaluated on three publicly available remote sensing image datasets:

  • Vaihingen dataset (ISPRS, 2019b): A drone-captured dataset of urban scenes with six categories.
  • Potsdam dataset (ISPRS, 2019a): An aerial image dataset of urban scenes, also with six categories.
  • BLU dataset (Ding et al., 2021): A satellite-based dataset from Beijing with six categories, including background/barren, built-up, vegetation, water, farmland, and road.

Key Insights and Ideas

  • Synergy of CNNs and Transformers: The paper strongly advocates for the combined use of CNNs and Transformers, demonstrating that their complementary strengths are crucial for high-performance semantic segmentation in complex remote sensing scenarios.
  • Dual-Branch for Comprehensive Context: The DGEM's dual-branch approach is key. It highlights that a single method for context extraction is insufficient; a combination of attention-based global context and pooling-based local context is more effective.
  • Inter-Layer Fusion is Crucial: The CIIM module underscores the importance of information flow and interaction between different feature levels in the decoder, not just within a single level.
  • Adaptive Fusion for Refinement: The FIGM module's adaptive fusion mechanism is critical for smoothing boundaries and reducing segmentation errors, suggesting that simple addition of features might not be optimal.

Key Results

The DGCFNet achieved state-of-the-art performance across the evaluated datasets:

  • Vaihingen Dataset: Achieved mIoU of 82.20%.
  • Potsdam Dataset: Achieved mIoU of 83.84%.
  • BLU Dataset: Achieved mIoU of 68.87%.

Ablation studies demonstrated the effectiveness of each proposed module:

  • Adding the DGEM module improved mIoU by 0.43% on the Potsdam dataset.
  • The CIIM module further improved performance, and using both CIIM modules (f12 and f23) was found to be necessary.
  • The FIGM module yielded the best overall results, with the final DGCFNet model achieving the highest scores across all categories and datasets. For instance, on the Potsdam dataset, the final model achieved a mean F1 of 91.08% and an mIoU of 83.84%.

Limitations and Future Work

The authors acknowledge a few limitations and suggest future research directions:

  • Small Target Segmentation: While the model excels at segmenting larger objects, its performance on small targets within complex backgrounds still has room for improvement, leading to boundary segmentation errors.
  • Model Lightweighting: The model consumes more computational resources than some smaller networks, and future work will focus on model lightweighting, potentially through techniques like model pruning in the decoder.
  • Future Work on Small Targets: The authors plan to explore representing object similarity using mutual relationship matrices between intermediate feature blocks and image patches to better segment small targets in complex backgrounds.

Summary updated

BibTeX
@article{liao2025dgcfnet,
  author = {Yuan Liao and Tongchi Zhou and Lu Li and Jinming Li and Jiuhao Shen and Askar Hamdulla},
  title = {{DGCFNet: Dual Global Context Fusion Network for remote sensing image semantic segmentation}},
  journal = {PeerJ Comput. Sci.},
  year = {2025},
  doi = {10.7717/peerj-cs.2786},
}
Figure