Abstract
Existing visual change detectors primarily focus on learning representations for changed regions between images. While effective, these methods often ignore the critical context provided by the unchanged background. This oversight limits the model's ability to distinguish between true structural changes and pseudo-changes caused by variations in sunlight, shadows, or sensor noise. In this work, we propose the Visual change Transformer (VcT), a model designed to mine common background information to learn consistent representations across image pairs. By utilizing a Reliable Token Mining (RTM) module and an Anchor-Primary Attention (APA) mechanism, VcT achieves state-of-the-art performance on multiple benchmarks while maintaining a remarkably low computational footprint.
Introduction
Remote sensing change detection (CD) involves identifying pixel-level differences between two images of the same geographic area captured at different times. This task is foundational for urban planning, disaster assessment, and environmental monitoring. However, the complexity of high-resolution (HSR) remote sensing data introduces significant challenges. "Pseudo-changes"—variations in lighting, atmospheric conditions, and soil moisture—often lead traditional models to produce false positives.
Most contemporary research focuses on enhancing the features of changed regions. VcT shifts this paradigm by arguing that the unchanged background context is the key to robust detection. By identifying and aligning consistent background features, the model can establish a reliable reference point, making it easier to isolate true semantic changes from environmental noise.
Methodology
The VcT architecture follows a multi-stage pipeline designed for both precision and efficiency.
1. Feature Extraction
The model employs a shared ResNet18 backbone to extract initial feature maps from the bi-temporal image pair. This ensures that both images are processed through a consistent representational space before interaction.
2. Reliable Token Mining (RTM)
The RTM module is the core innovation of VcT. To avoid the noise associated with random token selection, the model uses a Graph Neural Network (GNN) to generate a coarse change map. From this map, the model identifies "reliable" background tokens—pixels with the lowest probability of change. These tokens are then refined using K-means clustering. This two-stage process ensures that the selected tokens are not only reliable but also represent the diverse semantic categories of the background (e.g., roads, water, vegetation).
3. Transformer Interaction
The refined tokens undergo two stages of attention-based processing:
- Self-Attention: Models the internal relationships within each image's background tokens.
- Cross-Attention: Facilitates communication between the two time-steps, allowing the model to align background features across the temporal gap.
4. Anchor-Primary Attention (APA) and Prediction
The enhanced, globally-aware tokens act as "anchors" that are fused back into the original high-resolution feature maps via the APA module. Finally, a CNN-based prediction head calculates the difference between these enhanced features to produce the final binary change map.
Experimental Evaluation
VcT was evaluated on three major high-resolution remote sensing datasets: LEVIR-CD, WHU-CD, and DSIFN-CD. These datasets cover a range of challenges, from building changes in urban environments to post-earthquake disaster assessment.
Results and Efficiency
VcT achieved state-of-the-art (SOTA) results across all benchmarks:
- LEVIR-CD: Achieved an F1-score of 90.04% and an IoU of 81.89%.
- WHU-CD: Reached an F1-score of 89.58%.
Beyond accuracy, VcT is highly efficient. With only 3.57M parameters and a computational cost of 10.64 GFLOPs, it is significantly more lightweight than other SOTA models like IFNet (~50M parameters). Qualitative results demonstrate that VcT produces cleaner change maps with fewer fragmented noise pixels in unchanged areas, validating the effectiveness of its background-anchoring strategy.
Conclusion
The Visual change Transformer (VcT) demonstrates that mining unchanged background context is a powerful strategy for remote sensing change detection. By treating the background as a reliable anchor, VcT effectively mitigates the impact of pseudo-changes. Future work will explore adaptive token selection strategies and the integration of large-scale foundation models to further enhance semantic understanding in complex scenes.
Summary updated