VcT: Visual change Transformer for Remote Sensing Image Change Detection

Bo Jiang, Zitian Wang, Xixi Wang, Ziyan Zhang, Lan Chen, Xiao Wang, Bin Luo

Abstract

Existing visual change detectors primarily focus on learning representations for changed regions between images. While effective, these methods often ignore the critical context provided by the unchanged background. This oversight limits the model's ability to distinguish between true structural changes and pseudo-changes caused by variations in sunlight, shadows, or sensor noise. In this work, we propose the Visual change Transformer (VcT), a model designed to mine common background information to learn consistent representations across image pairs. By utilizing a Reliable Token Mining (RTM) module and an Anchor-Primary Attention (APA) mechanism, VcT achieves state-of-the-art performance on multiple benchmarks while maintaining a remarkably low computational footprint.

Introduction

Remote sensing change detection (CD) involves identifying pixel-level differences between two images of the same geographic area captured at different times. This task is foundational for urban planning, disaster assessment, and environmental monitoring. However, the complexity of high-resolution (HSR) remote sensing data introduces significant challenges. "Pseudo-changes"—variations in lighting, atmospheric conditions, and soil moisture—often lead traditional models to produce false positives.

Most contemporary research focuses on enhancing the features of changed regions. VcT shifts this paradigm by arguing that the unchanged background context is the key to robust detection. By identifying and aligning consistent background features, the model can establish a reliable reference point, making it easier to isolate true semantic changes from environmental noise.

Methodology

The VcT architecture follows a multi-stage pipeline designed for both precision and efficiency.

1. Feature Extraction

The model employs a shared ResNet18 backbone to extract initial feature maps from the bi-temporal image pair. This ensures that both images are processed through a consistent representational space before interaction.

2. Reliable Token Mining (RTM)

The RTM module is the core innovation of VcT. To avoid the noise associated with random token selection, the model uses a Graph Neural Network (GNN) to generate a coarse change map. From this map, the model identifies "reliable" background tokens—pixels with the lowest probability of change. These tokens are then refined using K-means clustering. This two-stage process ensures that the selected tokens are not only reliable but also represent the diverse semantic categories of the background (e.g., roads, water, vegetation).

3. Transformer Interaction

The refined tokens undergo two stages of attention-based processing:

  • Self-Attention: Models the internal relationships within each image's background tokens.
  • Cross-Attention: Facilitates communication between the two time-steps, allowing the model to align background features across the temporal gap.

4. Anchor-Primary Attention (APA) and Prediction

The enhanced, globally-aware tokens act as "anchors" that are fused back into the original high-resolution feature maps via the APA module. Finally, a CNN-based prediction head calculates the difference between these enhanced features to produce the final binary change map.

Experimental Evaluation

VcT was evaluated on three major high-resolution remote sensing datasets: LEVIR-CD, WHU-CD, and DSIFN-CD. These datasets cover a range of challenges, from building changes in urban environments to post-earthquake disaster assessment.

Results and Efficiency

VcT achieved state-of-the-art (SOTA) results across all benchmarks:

  • LEVIR-CD: Achieved an F1-score of 90.04% and an IoU of 81.89%.
  • WHU-CD: Reached an F1-score of 89.58%.

Beyond accuracy, VcT is highly efficient. With only 3.57M parameters and a computational cost of 10.64 GFLOPs, it is significantly more lightweight than other SOTA models like IFNet (~50M parameters). Qualitative results demonstrate that VcT produces cleaner change maps with fewer fragmented noise pixels in unchanged areas, validating the effectiveness of its background-anchoring strategy.

Conclusion

The Visual change Transformer (VcT) demonstrates that mining unchanged background context is a powerful strategy for remote sensing change detection. By treating the background as a reliable anchor, VcT effectively mitigates the impact of pseudo-changes. Future work will explore adaptive token selection strategies and the integration of large-scale foundation models to further enhance semantic understanding in complex scenes.

Summary updated

Abstract

Existing visual change detectors usually adopt CNNs or Transformers for feature representation learning and focus on learning effective representation for the changed regions between images. Although good performance can be obtained by enhancing the features of the change regions, however, these works are still limited mainly due to the ignorance of mining the unchanged background context information. It is known that one main challenge for change detection is how to obtain the consistent representations for two images involving different variations, such as spatial variation, sunlight intensity, etc. In this work, we demonstrate that carefully mining the common background information provides an important cue to learn the consistent representations for the two images which thus obviously facilitates the visual change detection problem. Based on this observation, we propose a novel Visual change Transformer (VcT) model for visual change detection problem. To be specific, a shared backbone network is first used to extract the feature maps for the given image pair. Then, each pixel of feature map is regarded as a graph node and the graph neural network is proposed to model the structured information for coarse change map prediction. Top-K reliable tokens can be mined from the map and refined by using the clustering algorithm. Then, these reliable tokens are enhanced by first utilizing self/cross-attention schemes and then interacting with original features via an anchor-primary attention learning module. Finally, the prediction head is proposed to get a more accurate change map. Extensive experiments on multiple benchmark datasets validated the effectiveness of our proposed VcT model.
BibTeX
@article{jiang2023vct,
  author = {Bo Jiang and Zitian Wang and Xixi Wang and Ziyan Zhang and Lan Chen and Xiao Wang and Bin Luo},
  title = {{VcT: Visual change Transformer for Remote Sensing Image Change Detection}},
  journal = {IEEE Trans. Geosci. Remote. Sens.},
  year = {2023},
  url = {https://openreview.net/forum?id=bXR57OEIt9},
}
Figure