TL;DR
This paper introduces a novel weakly supervised domain adaptation (WSDA) method for edge detection in high-resolution remote sensing images. By performing adversarial adaptation at both the edge and semantic levels, the method effectively transfers knowledge from a source domain with abundant data to a target domain with limited samples, significantly improving edge detection performance and reducing the need for extensive labeling.
What Problem Does This Paper Address?
Extracting precise edges of ground objects from high-resolution remote sensing imagery is crucial for applications like urban planning and change detection. While deep learning methods excel at this, they typically require large, high-quality labeled datasets. A major challenge arises when applying models trained on one dataset (source domain) to another with different characteristics (target domain), such as variations in ground objects, lighting, or imaging sources. This domain shift severely degrades performance. Consequently, creating new labeled datasets for each target domain is costly and time-consuming. This paper tackles the problem of reducing this dependence on target domain samples by enabling effective knowledge transfer from a source domain, specifically for edge detection tasks.
Methods
The core of this work is a weakly supervised domain adaptation (WSDA) approach built upon an adversarial learning framework. The proposed method consists of three main modules:
- Edge Detection Module (ED): This acts as the extractor, responsible for identifying semantic edges and outputting an "edge strength map" that indicates the confidence of predicted edges. The paper utilizes a modified D-LinkNet architecture called SEGOS, known for its ability to capture multi-scale information and maintain connectivity, which is crucial for edge detection.
- Edge Adaptation Module (EA): This module acts as a discriminator, performing adversarial domain adaptation at the edge level. It takes the edge strength maps from both source and target domains and tries to make them indistinguishable, thereby aligning low-level edge features.
- Semantic Adaptation Module (SA): This module also acts as a discriminator, performing adversarial domain adaptation at the semantic level. It utilizes the ground truth labels from the source domain and the edge strength maps from both domains to learn more robust semantic features. This is a key innovation, as many existing domain adaptation methods for segmentation overlook the importance of semantic alignment for edge tasks.
The adversarial learning process involves the ED module trying to generate features that can fool the EA and SA discriminators, while the discriminators try to distinguish between source and target domain features. Dynamic parameters are introduced to adjust the influence of source domain samples during different training stages, preventing overfitting in the early stages and fine-tuning for accuracy later on.
Datasets
The authors created two multi-category semantic edge datasets from high-resolution GF2 PMS imagery:
- Dataset 1: Yangyuan County, Hebei Province (500 images).
- Dataset 2: Jiashan County, Zhejiang Province (309 images).
These datasets were chosen because they exhibit significant differences in features and distribution, making them ideal for testing domain adaptation capabilities. The samples were labeled by different teams at different times, introducing potential variations in labeling standards. For experiments, a portion of samples from each dataset were used for testing, with the remainder used for training.
Key Insights and Ideas
- Domain Adaptation for Edge Detection is Novel: The paper highlights that applying domain adaptation specifically to edge detection tasks in remote sensing is a new area of research.
- Edge and Semantic Level Adaptation are Crucial: Simply adapting at the edge level can lead to overgeneralization and incorrect edge detections. The paper demonstrates that combining edge-level adaptation with semantic-level adaptation (using ground truth) is essential for robust performance.
- Dynamic Weighting for Training: Adjusting the weights of source domain samples during training is vital. Increasing their influence early on helps the network learn general features, while decreasing it later allows for fine-tuning on the target domain's specific characteristics.
- Pseudo-labeling Challenges for Edges: The paper notes that traditional self-training methods relying on pseudo-labels are difficult to apply effectively to edge detection due to the high accuracy and closure requirements of edges, where pseudo-labels often differ significantly from ground truth.
Key Results
- Significant Improvement with WSDA: The proposed WSDA method, combining edge and semantic adaptation, achieved superior performance compared to baseline methods and other domain adaptation techniques.
- Ablation Study Success: An ablation study showed that using either the edge adaptation module (EA) or the semantic adaptation module (SA) alone provided limited improvement. However, combining both modules (ED+EA+SA) resulted in the best performance, validating the necessity of both levels of adaptation.
- Quantitative Gains: The method achieved the highest ODS (Optimal Dataset Scale) and OIS (Optimal Image Scale) scores in experiments, outperforming fully supervised methods and other domain adaptation approaches. For instance, in one experiment, the proposed method achieved an ODS of 73.67% and OIS of 74.28% when using a small number of target domain samples, significantly better than SEGOS trained only on target domain samples (ODS 71.27%, OIS 72.99%).
- Effective with Few Samples: The method demonstrated strong performance even with a limited number of target domain samples, showing substantial improvements in accuracy as the number of target samples increased.
Limitations and Future Work
- Pixel Loss and Broken Lines: Despite its advancements, the paper acknowledges that the adversarial learning approach can still result in some lost pixels, leading to broken lines in the extracted edges.
- Pseudo-labeling Limitations: The authors suggest that if broken-line repairing methods could generate pseudo-labels more akin to human labels, self-training methods could potentially offer further improvements.
- Further Research on Self-Training: For complex tasks like multi-category semantic edge detection, further research into self-training learning is deemed necessary to overcome current limitations.
Summary updated