Revisiting Consistency Regularization for Semi-supervised Change Detection in Remote Sensing Images

Wele Gedara Chaminda Bandara, Vishal M. PatelarXiv 2022 · OpenReview

TL;DR

This paper introduces a novel semi-supervised approach for change detection in remote sensing images. By leveraging unlabeled data through a consistency regularization technique applied to feature differences, the method significantly boosts performance, achieving near-supervised results with only 10% of the labeled data.

What Problem Does This Paper Address?

Change Detection (CD) in remote sensing is crucial for applications like disaster assessment and urban monitoring. Existing deep learning methods for CD often require vast amounts of pixel-wise annotated data, which is expensive and time-consuming to acquire. While unlabeled remote sensing imagery is abundant, effectively utilizing it to improve CD performance has been a challenge. This paper aims to bridge this gap by proposing a semi-supervised framework that can learn from both limited labeled data and abundant unlabeled data.

Methods

The core of the proposed method is a semi-supervised learning framework that combines a standard supervised Cross-Entropy (CE) loss with a novel unsupervised consistency loss. The key idea is to enforce consistency in the model's predictions for unlabeled images. Instead of applying perturbations directly to the input images (which can introduce spurious changes), the authors apply small random perturbations to the deep feature difference map. This map is derived by subtracting the latent feature representations of pre-change and post-change images extracted by a Siamese encoder (based on ResNet50). The unsupervised loss then encourages the output change probability map to remain consistent under these perturbations of the feature difference map. This approach is inspired by the cluster assumption in semi-supervised learning, but the authors empirically show that this assumption holds in the feature difference domain, not the raw image domain, for CD tasks.

Datasets

The authors evaluate their method on two publicly available, widely-used building change detection datasets:

  • LEVIR: A dataset containing bi-temporal remote sensing images.
  • WHU: Another dataset for building change detection.

They create non-overlapping patches of size 256x256 for training and testing. For semi-supervised experiments, they randomly select a percentage of labeled training data and use the rest as unlabeled data.

Key Insights and Ideas

  • Feature Difference Domain is Key: The "cluster assumption" (that decision boundaries lie in low-density regions) is crucial for semi-supervised learning. The paper demonstrates this assumption doesn't hold in the raw bi-temporal image domain for CD but does hold in the deep feature difference domain. This insight is fundamental to their approach.
  • Perturbing Features, Not Images: Applying perturbations to the feature difference map, rather than the raw images, is a more robust way to enforce consistency without introducing artificial changes that could confuse the model.
  • Efficiency with Little Data: The method shows remarkable performance gains even with very limited labeled data (as little as 5-10%), significantly outperforming existing semi-supervised methods.
  • Improved Generalization: The consistency regularization approach makes the model more robust to variations, leading to better generalization performance, even on out-of-distribution datasets.

Key Results

  • Near-Supervised Performance: With only 10% of the labeled data, the proposed method achieves performance (IoU) very close to the fully supervised method that uses 100% of the labeled data. For instance, on LEVIR, using 10% labeled data, their method achieves an IoU of 76.22%, while the supervised method with 100% data reaches 77.16%.
  • Outperforming SOTA: The proposed semi-supervised CD method significantly outperforms existing SOTA semi-supervised CD methods (like SemiCDNet) by a considerable margin, especially when labeled data is scarce. For example, with 5% labeled data on LEVIR, their method achieves a 5% higher IoU than the SOTA.
  • Cross-Dataset Performance: The model demonstrates strong generalizability. When trained on one dataset (e.g., LEVIR) and tested on another (e.g., WHU), it consistently outperforms supervised-only methods and other semi-supervised approaches, highlighting its robustness.
  • Qualitative Improvements: Visual results show that the proposed method generates cleaner and more accurate change maps, better identifying subtle changes and reducing false positives compared to other methods.

Limitations and Future Work

  • Assumptions: While the paper argues against assumptions about unlabeled data distribution for GAN-based methods, their approach still relies on the cluster assumption holding in the feature difference domain.
  • Future Work: The paper's success with consistency regularization on feature differences opens avenues for exploring more sophisticated perturbation strategies and loss functions within this framework. The authors also suggest that the robustness gained from this method could be beneficial for other remote sensing tasks.

Summary updated

Abstract

Remote-sensing (RS) Change Detection (CD) aims to detect "changes of interest" from co-registered bi-temporal images. The performance of existing deep supervised CD methods is attributed to the large amounts of annotated data used to train the networks. However, annotating large amounts of remote sensing images is labor-intensive and expensive, particularly with bi-temporal images, as it requires pixel-wise comparisons by a human expert. On the other hand, we often have access to unlimited unlabeled multi-temporal RS imagery thanks to ever-increasing earth observation programs. In this paper, we propose a simple yet effective way to leverage the information from unlabeled bi-temporal images to improve the performance of CD approaches. More specifically, we propose a semi-supervised CD model in which we formulate an unsupervised CD loss in addition to the supervised Cross-Entropy (CE) loss by constraining the output change probability map of a given unlabeled bi-temporal image pair to be consistent under the small random perturbations applied on the deep feature difference map that is obtained by subtracting their latent feature representations. Experiments conducted on two publicly available CD datasets show that the proposed semi-supervised CD method can reach closer to the performance of supervised CD even with access to as little as 10% of the annotated training data. Code available at https://github.com/wgcban/SemiCD
BibTeX
@misc{bandara2022revisiting,
  author = {Wele Gedara Chaminda Bandara and Vishal M. Patel},
  title = {{Revisiting Consistency Regularization for Semi-supervised Change Detection in Remote Sensing Images}},
  year = {2022},
  eprint = {2204.08454},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2204.08454},
}
Figure