TransY-Net: Learning Fully Transformer Networks for Change Detection of Remote Sensing Images

Tianyu Yan, Zifu Wan, Pingping Zhang, Gong Cheng, Huchuan Lu

TL;DR

TransY-Net is a novel, fully Transformer-based "Y-shaped" network designed to accurately detect changes in remote sensing images. By leveraging Swin Transformers for global context and a multi-scale attention pyramid for boundary refinement, it eliminates the common issues of incomplete detection regions and irregular boundaries, setting new state-of-the-art benchmarks across both optical and SAR datasets.

What Problem Does This Paper Address?

Change Detection (CD) is a cornerstone of remote sensing, used for everything from urban planning to disaster monitoring. The goal is simple: take two images of the same location from different times and highlight what has changed.

However, current deep learning methods—mostly based on Convolutional Neural Networks (CNNs)—suffer from two major flaws:

  1. Incomplete Regions: Because CNNs focus on local textures, they often fail to recognize a large object (like a building) as a single unit, resulting in "holes" in the detection map.
  2. Irregular Boundaries: Complex scenes with shadows and varying light lead to jagged, inaccurate edges that don't match the real-world shapes of objects.

Furthermore, many models fail to distinguish "pseudo-changes" (like a moving shadow or seasonal vegetation shifts) from actual structural changes.

Methods

The authors propose TransY-Net, a "Y-shape" architecture consisting of four main components:

  • Siamese Feature Extraction (SFE): Instead of standard convolutions, the model uses weight-shared Swin Transformers as the backbone. Transformers excel at modeling "long-range dependencies," meaning the model looks at the whole image context to decide if a pixel has changed, rather than just its immediate neighbors.
  • Deep Feature Enhancement (DFE): Most models just subtract the features of the two images. TransY-Net performs both summation (to highlight commonalities) and difference (to highlight changes). It also introduces "contrast features" using multi-scale average pooling to help change regions stand out from their background.
  • Progressive Change Prediction (PCP): This is a decoder pyramid that integrates features from different scales. It uses a Progressive Attention Module (PAM) that applies both spatial and channel-level attention to refine the boundaries of the detected regions.
  • Hybrid Loss Function: To train the model, the authors use a combination of three losses: Weighted Binary Cross-Entropy (to handle class imbalance), SSIM (to maintain structural integrity), and soft IoU (to ensure global overlap).

Datasets

The model was evaluated on six diverse benchmarks:

  • Optical Imagery: LEVIR-CD, WHU-CD, SYSU-CD, and the challenging Google-CD.
  • SAR (Synthetic Aperture Radar) Imagery: Ottawa (flood detection) and Sulzberger (ice shelf breakup).

The inclusion of SAR datasets is particularly noteworthy, as SAR data is much noisier and harder to process than standard optical photos.

Key Insights and Ideas

  • Global Context is King: The "aha moment" here is that the "holes" in change detection are a symptom of a limited receptive field. By using Transformers, the model "understands" the entire object, leading to much more complete and solid detection masks.
  • Summation Matters: While it seems counterintuitive to add features when looking for differences, the authors show that summation helps the model understand the underlying scene structure, which in turn makes the detected changes more accurate.
  • Boundary-Aware Supervision: By specifically weighting the loss function to care more about pixels near the edges (using gradient operators), the model learns to produce sharp, clean boundaries that look much more like human-drawn maps.

Key Results

TransY-Net achieved state-of-the-art (SOTA) performance across the board:

  • LEVIR-CD: Achieved an F1 score of 91.90% and an IoU of 83.64%, outperforming previous Transformer-based models like BIT and ChangeFormer.
  • WHU-CD: Reached a staggering 93.38% F1 score, a significant improvement over the previous best methods.
  • Generalization: The model successfully localized changes in disaster-stricken SAR environments (floods and ice breakups), proving it isn't just "overfitting" to pretty optical photos.
  • Visual Quality: Qualitative results show that TransY-Net produces much cleaner, more rectangular building footprints compared to the "blobby" outputs of earlier CNN models.

Limitations and Future Work

The authors acknowledge a few hurdles:

  • Computational Complexity: Transformers are computationally expensive. While Swin Transformers help mitigate this, the model still requires significant GPU resources compared to lightweight CNNs.
  • Data Hunger: Like most deep learning models, TransY-Net requires large amounts of densely labeled data, which is expensive and slow to produce in the remote sensing field.

Future Directions: The team plans to explore more efficient Transformer structures to reduce the computational load and investigate unsupervised or weakly-supervised learning methods to reduce the reliance on massive, hand-labeled datasets.

Summary updated

Abstract

In the remote sensing field, Change Detection (CD) aims to identify and localize the changed regions from dual-phase images over the same places. Recently, it has achieved great progress with the advances of deep learning. However, current methods generally deliver incomplete CD regions and irregular CD boundaries due to the limited representation ability of the extracted visual features. To relieve these issues, in this work we propose a novel Transformer-based learning framework named TransY-Net for remote sensing image CD, which improves the feature extraction from a global view and combines multi-level visual features in a pyramid manner. More specifically, the proposed framework first utilizes the advantages of Transformers in long-range dependency modeling. It can help to learn more discriminative global-level features and obtain complete CD regions. Then, we introduce a novel pyramid structure to aggregate multi-level visual features from Transformers for feature enhancement. The pyramid structure grafted with a Progressive Attention Module (PAM) can improve the feature representation ability with additional inter-dependencies through spatial and channel attentions. Finally, to better train the whole framework, we utilize the deeply-supervised learning with multiple boundary-aware loss functions. Extensive experiments demonstrate that our proposed method achieves a new state-of-the-art performance on four optical and two SAR image CD benchmarks. The source code is released at https://github.com/Drchip61/TransYNet.
BibTeX
@article{yan2023transy,
  author = {Tianyu Yan and Zifu Wan and Pingping Zhang and Gong Cheng and Huchuan Lu},
  title = {{TransY-Net: Learning Fully Transformer Networks for Change Detection of Remote Sensing Images}},
  journal = {IEEE Trans. Geosci. Remote. Sens.},
  year = {2023},
  doi = {10.1109/tgrs.2023.3327253},
}

Loading related content...

Figure