ICD: VHR-Oriented Interactive Change-Detection Algorithm

Zhuoran Jiang, Xinxin Zhou, Wei Cao, Zaihong Sun, Changbin WuISPRS Int. J. Geo Inf 2022 · DOI

TL;DR

This paper introduces an interactive change-detection algorithm (ICD) for very high-resolution (VHR) remote-sensing images. By allowing users to provide positive and negative clicks to correct initial predictions, the ICD algorithm significantly improves detection accuracy and efficiency, addressing limitations of purely automated deep learning methods.

What Problem Does This Paper Address?

Change detection, the process of identifying differences between images of the same area taken at different times, is crucial for applications like disaster assessment and land-use monitoring. While deep learning has revolutionized this field, achieving high accuracy and speed, current automated methods struggle with local prediction defects, leading to incomplete or inaccurate change maps. Traditional manual correction is time-consuming and labor-intensive. This paper aims to bridge this gap by developing an interactive deep learning approach that leverages user input to refine change detection results efficiently.

Methods

The core of the proposed solution is the Interactive Change-Detection (ICD) network. This network builds upon a deep convolutional neural network (specifically, a ResNet18 backbone) and integrates user-provided positive and negative clicks into the change detection process.

Here's a breakdown of the key methodological aspects:

  • Interactive Click Integration: Unlike traditional methods that rely solely on automated feature extraction, ICD incorporates user clicks. Positive clicks highlight areas that should be detected as changed but were missed, while negative clicks mark areas incorrectly identified as changed.
  • Distance Layer Fusion: The clicks are not simply overlaid but are fused with the distance map (which represents preliminary change information) within the network. This fusion is achieved through convolutional operations and weighted addition, allowing the network to learn from both the image features and the user's guidance.
  • Secondary Feature Extraction: After fusing click information with the distance map, the network performs a secondary feature extraction step. This allows the model to re-process the combined information, further enhancing the correction capabilities.
  • Automatic Click Generation (for training): To train the ICD network effectively, an automatic click generation algorithm is proposed. This algorithm uses the model's predictions from the previous training iteration and compares them with ground truth labels to identify false negatives and false positives, generating synthetic clicks to guide the learning process. This is crucial for creating a robust interactive system.
  • Gradient Clicks: The paper explores different ways to represent clicks, finding that gradient points (where the click's influence fades with distance) are more effective than simple solid points for improving correction accuracy and stability.

Datasets

The proposed ICD algorithm was evaluated on two publicly available datasets:

  • WHU Building Dataset: This dataset consists of aerial images from Christchurch, New Zealand, captured in 2011 and 2016, with a resolution of 0.2m. The original images were clipped into 256x256 samples. Data augmentation (rotations) was applied to increase the dataset size.
  • LEVIR-CD Dataset: This dataset comprises remote-sensing images from Texas with a time span from 2002 to 2018 and a resolution of 0.5m. Similar to the WHU dataset, images were clipped to 256x256, and data augmentation was used.

Key Insights and Ideas

  • Human-in-the-Loop for Deep Learning: The paper demonstrates that integrating human interaction (clicks) into deep learning models can significantly overcome the limitations of purely automated systems, especially for complex tasks like change detection in VHR imagery.
  • Clicks as Refined Input: Clicks are not just post-processing corrections but are integrated within the network's feature extraction pipeline, allowing the model to adapt its internal representations based on user feedback.
  • Gradient Clicks Offer Superiority: Using gradient-based representations for clicks, rather than simple binary masks, leads to more stable and accurate correction results. This suggests that the degree of correction matters, not just its presence.
  • Secondary Feature Extraction is Key: Performing feature extraction after fusing click information is crucial for maximizing the impact of user input and improving correction performance.

Key Results

The ICD algorithm, particularly the Gradient_Res variant (using gradient clicks and secondary feature extraction), showed significant improvements:

  • F1-Score Improvement: On the WHU dataset, the F1-score increased by 4.3% with 10 clicks compared to the original network. On the LEVIR-CD dataset, the improvement was 4.2%.
  • Click Correction Range (CR):
    • Positive clicks (filling missed changes) achieved CRs of 0.779 (WHU) and 0.815 (LEVIR-CD) with 10 clicks, indicating that positive clicks can effectively fill approximately 80% of detection holes.
  • Negative clicks (removing false positives) achieved CRs of 0.546 (WHU) and 0.655 (LEVIR-CD), meaning negative clicks can eliminate about 50-65% of false detection areas.
  • Comparison of Click Types: Gradient clicks consistently outperformed solid clicks in terms of correction accuracy and stability.
  • Impact of Secondary Feature Extraction: The Gradient_Res model (with secondary feature extraction) showed better performance than Gradient_No_Res (without it), highlighting the importance of re-processing features after click integration.

Limitations and Future Work

  • User Judgment Dependency: The effectiveness of ICD relies on the user's ability to provide accurate clicks. Inaccurate or misplaced clicks can negatively impact the detection results, especially for negative clicks.
  • Click Influence Discrepancy: Positive clicks appear to have a stronger correction influence than negative clicks.
  • Boundary Correction Difficulty: Correcting areas at the boundary of change regions can be challenging, and inaccurate clicks can affect the boundary shape.

Future work directions suggested by the authors include:

  • Optimizing Click Performance: Developing methods to minimize the negative impact of incorrect clicks.
  • Improving Point Selection: Enhancing the automatic point selection algorithm to better mimic human selection and achieve results closer to manual correction accuracy.

Summary updated

BibTeX
@article{jiang2022icd,
  author = {Zhuoran Jiang and Xinxin Zhou and Wei Cao and Zaihong Sun and Changbin Wu},
  title = {{ICD: VHR-Oriented Interactive Change-Detection Algorithm}},
  journal = {ISPRS Int. J. Geo Inf.},
  year = {2022},
  doi = {10.3390/ijgi11100503},
}

Loading related content...

Figure