CCTNet: CNN and Cross-Shaped Transformer Hybrid Network for Remote Sensing Image Semantic Segmentation

Honglin Wu, Zhitao Zeng, Peng Huang, Xinyu Yu, Min ZhangIEEE JSTARS 2024 · DOI

Abstract

Deep learning methods have achieved great success in the field of remote sensing image segmentation in recent years, but building a lightweight segmentation model with comprehensive local and global feature extraction capabilities remains a challenging task. In this article, we propose a convolutional neural network (CNN) and cross-shaped transformer hybrid network (CCTNet) for semantic segmentation of high-resolution remote sensing images. This model follows an encoder–decoder structure. It employs ResNet18 as an encoder to extract hierarchical feature information, and constructs a transformer decoder based on efficient cross-shaped self-attention to fully model local and global feature information and achieve lightweighting of the network. Moreover, the transformer block introduces a mixed-scale convolutional feedforward network to further enhance multiscale information extraction. Furthermore, a simplified and efficient feature aggregation module is leveraged to gradually aggregate local and global information at different stages. Extensive comparison experiments on the ISPRS Vaihingen and Potsdam datasets reveal that our method obtains superior performance compared with state-of-the-art lightweight methods.
BibTeX
@article{wu2024cctnet,
  author = {Honglin Wu and Zhitao Zeng and Peng Huang and Xinyu Yu and Min Zhang},
  title = {{CCTNet: CNN and Cross-Shaped Transformer Hybrid Network for Remote Sensing Image Semantic Segmentation}},
  journal = {IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing},
  year = {2024},
  doi = {10.1109/jstars.2024.3487003},
}

Loading related content...

Figure