A Deep Learning Based Platform for Remote Sensing Images Change Detection Integrating Crowdsourcing and Active Learning

Zhibao Wang, Jie Zhang, Lu Bai, Huan Chang, Yuanlin Chen, Ying Zhang, Jinhua Tao2024 · DOI

TL;DR

This paper introduces a novel platform for remote sensing image change detection that combines deep learning with crowdsourcing and active learning. This approach addresses the critical challenge of limited annotated data in remote sensing by creating a human-machine collaborative system that improves model generalization and efficiency.

What Problem Does This Paper Address?

Detecting changes in remote sensing images is crucial for monitoring land use, natural disasters, and urban development. Traditional methods often rely on manual interpretation and on-site verification, which are time-consuming, labor-intensive, and struggle to meet the timeliness requirements of real-world applications. While deep learning has shown promise, its effectiveness is heavily dependent on large, high-quality annotated datasets, which are scarce and expensive to create in the remote sensing domain. This paper tackles the problem of weak model generalization and the high cost of data annotation by proposing an integrated framework that leverages crowdsourcing and active learning to create a more efficient and intelligent change detection system.

Methods

The core of this work is a platform that integrates three key components:

  • Deep Learning-Based Change Detection: The platform utilizes deep learning models (e.g., U-Net, U-Net++, LinkNet, DeepLabV3plus, FPN) for the actual change detection task. These models are capable of automatically learning complex features from remote sensing imagery.
  • Crowdsourcing: To overcome the data annotation bottleneck, the platform employs crowdsourcing. This involves distributing the task of labeling remote sensing images to a large pool of online workers. To manage data quality, the paper proposes an Annotator Qualification Assessment Model and a Quality Control Model. The former categorizes annotators based on their performance (e.g., professional, excellent, good, adequate, average, unqualified) using a similarity threshold (K) derived from gold standard test data and Euclidean distance. The latter uses algorithms like Majority Voting (MV) and Expectation-Maximization (EM) to aggregate and validate annotations.
  • Active Learning and Human-in-the-Loop: To further optimize the learning process and reduce annotation costs, the platform incorporates active learning and human-in-the-loop techniques. Active learning strategies (Least Confidence, Entropy Sampling, Committee Voting Sampling) intelligently select the most informative unlabeled samples for annotation. The human-in-the-loop aspect ensures continuous interaction between human annotators and the machine learning model, allowing for iterative refinement of annotations and model training. This collaborative approach aims to make the model more adaptable to complex scenes and improve its generalization ability.

Datasets

The paper uses two publicly available datasets for evaluation:

  • CDD (Change Detection Dataset): This dataset contains 16,000 images with spatial resolutions ranging from 3 to 100 cm. It captures seasonal changes and variations in objects like cars, buildings, and natural landscapes.
  • LEVIR-CD (Large-scale Building Change Detection Dataset): This dataset focuses on architectural changes and includes 31,333 building change patches from high-resolution remote sensing images (0.5m resolution) captured between 2002 and 2018.

Key Insights and Ideas

  • Human-Machine Collaboration is Key: The paper strongly advocates for a synergistic approach where human intelligence (domain expertise, annotation) and machine learning (automated detection, feature extraction) work together. This is crucial for tackling the complexities of remote sensing data.
  • Active Learning Significantly Reduces Annotation Burden: By intelligently selecting samples, active learning methods (especially committee voting) demonstrated the ability to achieve high detection accuracy with substantially fewer labeled samples compared to random sampling. This is a critical finding for practical applications where labeling is a major bottleneck.
  • Crowdsourcing Quality Control is Essential: Simply using crowdsourcing isn't enough. The proposed annotator qualification and quality control mechanisms are vital for ensuring that the aggregated annotations are reliable and useful for training robust models. The EM algorithm showed better performance than MV in quality assessment.
  • Platform Integration is Crucial: The paper emphasizes the need for a unified platform that seamlessly integrates data management, annotation tools, crowdsourcing workflows, quality control, and model training. This holistic approach streamlines the entire change detection pipeline.

Key Results

  • The proposed platform successfully integrates crowdsourcing, active learning, and human-in-the-loop techniques for remote sensing change detection.
  • The Annotator Qualification Assessment Model effectively categorized annotators, with a threshold K of 0.35 identifying unqualified annotators.
  • The EM algorithm outperformed the MV algorithm in estimating annotation quality, demonstrating its suitability for crowdsourced image segmentation.
  • Active learning methods, particularly Committee Voting Sampling, achieved comparable or better F1-scores with fewer samples compared to random sampling, indicating significant efficiency gains. For instance, with 9000 samples, Committee Voting Sampling achieved an F1-score of approximately 0.85, while random sampling was around 0.79.
  • The developed platform provides a user-friendly interface for managing datasets, annotating images, and training models, showcasing visualization of sample libraries and training metrics.

Limitations and Future Work

  • Generalization to Diverse Scenarios: While the platform aims for broad applicability, the paper acknowledges that adapting to new, unseen scenarios or different types of change detection tasks might still pose challenges.
  • Annotator Fatigue and Bias: The paper touches upon the variability in annotator quality and attitudes. Future work could explore more sophisticated methods to mitigate the impact of annotator fatigue or biases.
  • Scalability of Quality Control: While the EM and MV algorithms are presented, further research into optimizing these for extremely large-scale crowdsourcing operations could be beneficial.
  • Real-time Performance: The paper focuses on improving accuracy and efficiency. Future work could explore optimizing the platform for near real-time change detection applications.

Summary updated

BibTeX
@misc{wang2024deep,
  author = {Zhibao Wang and Jie Zhang and Lu Bai and Huan Chang and Yuanlin Chen and Ying Zhang and Jinhua Tao},
  title = {{A Deep Learning Based Platform for Remote Sensing Images Change Detection Integrating Crowdsourcing and Active Learning}},
  year = {2024},
  doi = {10.3390/s24051509},
}
Figure