Enhanced Semantic Information Transfer of Multi-Domain Samples: An Adversarial Edge Detection Method Using Few High-Resolution Remote Sensing Images

Liegang Xia, Dezhi Yang, Junxia Zhang, Haiping Yang, Jun ChenSensors 2022 · DOI

TL;DR

This paper introduces a novel weakly supervised domain adaptation (WSDA) method for edge detection in high-resolution remote sensing images. By performing adversarial adaptation at both the edge and semantic levels, the method effectively transfers knowledge from a source domain with abundant data to a target domain with limited samples, significantly improving edge detection performance and reducing the need for extensive labeling.

What Problem Does This Paper Address?

Extracting precise edges of ground objects from high-resolution remote sensing imagery is crucial for applications like urban planning and change detection. While deep learning methods excel at this, they typically require large, high-quality labeled datasets. A major challenge arises when applying models trained on one dataset (source domain) to another with different characteristics (target domain), such as variations in ground objects, lighting, or imaging sources. This domain shift severely degrades performance. Consequently, creating new labeled datasets for each target domain is costly and time-consuming. This paper tackles the problem of reducing this dependence on target domain samples by enabling effective knowledge transfer from a source domain, specifically for edge detection tasks.

Methods

The core of this work is a weakly supervised domain adaptation (WSDA) approach built upon an adversarial learning framework. The proposed method consists of three main modules:

  1. Edge Detection Module (ED): This acts as the extractor, responsible for identifying semantic edges and outputting an "edge strength map" that indicates the confidence of predicted edges. The paper utilizes a modified D-LinkNet architecture called SEGOS, known for its ability to capture multi-scale information and maintain connectivity, which is crucial for edge detection.
  2. Edge Adaptation Module (EA): This module acts as a discriminator, performing adversarial domain adaptation at the edge level. It takes the edge strength maps from both source and target domains and tries to make them indistinguishable, thereby aligning low-level edge features.
  3. Semantic Adaptation Module (SA): This module also acts as a discriminator, performing adversarial domain adaptation at the semantic level. It utilizes the ground truth labels from the source domain and the edge strength maps from both domains to learn more robust semantic features. This is a key innovation, as many existing domain adaptation methods for segmentation overlook the importance of semantic alignment for edge tasks.

The adversarial learning process involves the ED module trying to generate features that can fool the EA and SA discriminators, while the discriminators try to distinguish between source and target domain features. Dynamic parameters are introduced to adjust the influence of source domain samples during different training stages, preventing overfitting in the early stages and fine-tuning for accuracy later on.

Datasets

The authors created two multi-category semantic edge datasets from high-resolution GF2 PMS imagery:

  • Dataset 1: Yangyuan County, Hebei Province (500 images).
  • Dataset 2: Jiashan County, Zhejiang Province (309 images).

These datasets were chosen because they exhibit significant differences in features and distribution, making them ideal for testing domain adaptation capabilities. The samples were labeled by different teams at different times, introducing potential variations in labeling standards. For experiments, a portion of samples from each dataset were used for testing, with the remainder used for training.

Key Insights and Ideas

  • Domain Adaptation for Edge Detection is Novel: The paper highlights that applying domain adaptation specifically to edge detection tasks in remote sensing is a new area of research.
  • Edge and Semantic Level Adaptation are Crucial: Simply adapting at the edge level can lead to overgeneralization and incorrect edge detections. The paper demonstrates that combining edge-level adaptation with semantic-level adaptation (using ground truth) is essential for robust performance.
  • Dynamic Weighting for Training: Adjusting the weights of source domain samples during training is vital. Increasing their influence early on helps the network learn general features, while decreasing it later allows for fine-tuning on the target domain's specific characteristics.
  • Pseudo-labeling Challenges for Edges: The paper notes that traditional self-training methods relying on pseudo-labels are difficult to apply effectively to edge detection due to the high accuracy and closure requirements of edges, where pseudo-labels often differ significantly from ground truth.

Key Results

  • Significant Improvement with WSDA: The proposed WSDA method, combining edge and semantic adaptation, achieved superior performance compared to baseline methods and other domain adaptation techniques.
  • Ablation Study Success: An ablation study showed that using either the edge adaptation module (EA) or the semantic adaptation module (SA) alone provided limited improvement. However, combining both modules (ED+EA+SA) resulted in the best performance, validating the necessity of both levels of adaptation.
  • Quantitative Gains: The method achieved the highest ODS (Optimal Dataset Scale) and OIS (Optimal Image Scale) scores in experiments, outperforming fully supervised methods and other domain adaptation approaches. For instance, in one experiment, the proposed method achieved an ODS of 73.67% and OIS of 74.28% when using a small number of target domain samples, significantly better than SEGOS trained only on target domain samples (ODS 71.27%, OIS 72.99%).
  • Effective with Few Samples: The method demonstrated strong performance even with a limited number of target domain samples, showing substantial improvements in accuracy as the number of target samples increased.

Limitations and Future Work

  • Pixel Loss and Broken Lines: Despite its advancements, the paper acknowledges that the adversarial learning approach can still result in some lost pixels, leading to broken lines in the extracted edges.
  • Pseudo-labeling Limitations: The authors suggest that if broken-line repairing methods could generate pseudo-labels more akin to human labels, self-training methods could potentially offer further improvements.
  • Further Research on Self-Training: For complex tasks like multi-category semantic edge detection, further research into self-training learning is deemed necessary to overcome current limitations.

Summary updated

Abstract

Edge detection of ground objects is a typical task in the field of remote sensing and has advantages in accomplishing many complex ground object extraction tasks. Although recent mainstream edge detection methods based on deep learning have significant effects, these methods have a very high dependence on the quantity and quality of samples. Moreover, using datasets from other domains in detection tasks often leads to degraded network performance due to variations in the ground objects in different regions. If this problem can be solved to allow datasets from other domains to be reused, the number of labeled samples required in the new task domain can be reduced, thereby shortening the task cycle and reducing task costs. In this paper, we propose a weakly supervised domain adaptation method to address the high dependence of edge extraction networks on samples. The domain adaptation is performed on the edge level and the semantic level, which prevents deviations in the semantic features that are caused by the overgeneralization of edge features. Additionally, the effectiveness of our proposed domain adaptation module is verified. Finally, we demonstrate the superior edge extraction performance of our method in the SEGOS edge extraction network in contrast to other edge extraction methods.
BibTeX
@inproceedings{xia2022enhanced,
  author = {Liegang Xia and Dezhi Yang and Junxia Zhang and Haiping Yang and Jun Chen},
  title = {{Enhanced Semantic Information Transfer of Multi-Domain Samples: An Adversarial Edge Detection Method Using Few High-Resolution Remote Sensing Images}},
  booktitle = {Sensors},
  year = {2022},
  doi = {10.3390/s22155678},
}
Figure