TL;DR
This paper introduces a novel semi-supervised approach for change detection in remote sensing images. By leveraging unlabeled data through a consistency regularization technique applied to feature differences, the method significantly boosts performance, achieving near-supervised results with only 10% of the labeled data.
What Problem Does This Paper Address?
Change Detection (CD) in remote sensing is crucial for applications like disaster assessment and urban monitoring. Existing deep learning methods for CD often require vast amounts of pixel-wise annotated data, which is expensive and time-consuming to acquire. While unlabeled remote sensing imagery is abundant, effectively utilizing it to improve CD performance has been a challenge. This paper aims to bridge this gap by proposing a semi-supervised framework that can learn from both limited labeled data and abundant unlabeled data.
Methods
The core of the proposed method is a semi-supervised learning framework that combines a standard supervised Cross-Entropy (CE) loss with a novel unsupervised consistency loss. The key idea is to enforce consistency in the model's predictions for unlabeled images. Instead of applying perturbations directly to the input images (which can introduce spurious changes), the authors apply small random perturbations to the deep feature difference map. This map is derived by subtracting the latent feature representations of pre-change and post-change images extracted by a Siamese encoder (based on ResNet50). The unsupervised loss then encourages the output change probability map to remain consistent under these perturbations of the feature difference map. This approach is inspired by the cluster assumption in semi-supervised learning, but the authors empirically show that this assumption holds in the feature difference domain, not the raw image domain, for CD tasks.
Datasets
The authors evaluate their method on two publicly available, widely-used building change detection datasets:
- LEVIR: A dataset containing bi-temporal remote sensing images.
- WHU: Another dataset for building change detection.
They create non-overlapping patches of size 256x256 for training and testing. For semi-supervised experiments, they randomly select a percentage of labeled training data and use the rest as unlabeled data.
Key Insights and Ideas
- Feature Difference Domain is Key: The "cluster assumption" (that decision boundaries lie in low-density regions) is crucial for semi-supervised learning. The paper demonstrates this assumption doesn't hold in the raw bi-temporal image domain for CD but does hold in the deep feature difference domain. This insight is fundamental to their approach.
- Perturbing Features, Not Images: Applying perturbations to the feature difference map, rather than the raw images, is a more robust way to enforce consistency without introducing artificial changes that could confuse the model.
- Efficiency with Little Data: The method shows remarkable performance gains even with very limited labeled data (as little as 5-10%), significantly outperforming existing semi-supervised methods.
- Improved Generalization: The consistency regularization approach makes the model more robust to variations, leading to better generalization performance, even on out-of-distribution datasets.
Key Results
- Near-Supervised Performance: With only 10% of the labeled data, the proposed method achieves performance (IoU) very close to the fully supervised method that uses 100% of the labeled data. For instance, on LEVIR, using 10% labeled data, their method achieves an IoU of 76.22%, while the supervised method with 100% data reaches 77.16%.
- Outperforming SOTA: The proposed semi-supervised CD method significantly outperforms existing SOTA semi-supervised CD methods (like SemiCDNet) by a considerable margin, especially when labeled data is scarce. For example, with 5% labeled data on LEVIR, their method achieves a 5% higher IoU than the SOTA.
- Cross-Dataset Performance: The model demonstrates strong generalizability. When trained on one dataset (e.g., LEVIR) and tested on another (e.g., WHU), it consistently outperforms supervised-only methods and other semi-supervised approaches, highlighting its robustness.
- Qualitative Improvements: Visual results show that the proposed method generates cleaner and more accurate change maps, better identifying subtle changes and reducing false positives compared to other methods.
Limitations and Future Work
- Assumptions: While the paper argues against assumptions about unlabeled data distribution for GAN-based methods, their approach still relies on the cluster assumption holding in the feature difference domain.
- Future Work: The paper's success with consistency regularization on feature differences opens avenues for exploring more sophisticated perturbation strategies and loss functions within this framework. The authors also suggest that the robustness gained from this method could be beneficial for other remote sensing tasks.
Summary updated