Debiased Learning for Remote Sensing Data

Chun-Hsiao Yeh, Xudong Wang, Stella X. Yu, Charles Hill, Zackery Steck, Scott Kangas, Aaron A. ReitearXiv 2023 · OpenReview

TL;DR

This paper introduces a semi-supervised learning approach specifically designed for remote sensing data, which often suffers from limited annotations and class imbalances. By adapting the FixMatch framework with domain-specific augmentations and a novel debiasing technique for pseudo-labels, the authors achieve significant accuracy improvements over existing methods, even with a small fraction of labeled data.

What Problem Does This Paper Address?

Deep learning models thrive on large, well-annotated datasets like ImageNet. However, remote sensing data presents a unique challenge: while abundant, it's sparsely annotated due to the difficulty and cost of manual labeling, especially given the uncommon viewing angles and ambiguous patterns. This scarcity hinders the direct application of supervised learning. Existing transfer learning approaches, while useful for other domains, struggle with the high-resolution details and class imbalance inherent in remote sensing imagery. Semi-supervised learning (SSL) offers a promising avenue by leveraging both labeled and unlabeled data, but standard SSL techniques like FixMatch require careful adaptation to the specific characteristics of remote sensing data, particularly its inherent biases.

Methods

The core of this work lies in two key contributions:

  1. Domain-Specific Augmentations for FixMatch: The authors adapt the FixMatch SSL framework, which uses consistency regularization between weakly and strongly augmented views of unlabeled data to generate pseudo-labels. They meticulously design robust "strong" and "weak" augmentations tailored for remote sensing. Instead of generic augmentations like RandAugment (which can introduce artifacts in remote sensing imagery), they opt for a combination of random resize cropping, horizontal flipping, rotation, and scaling. These augmentations are chosen to simulate variations in imaging angles and ground sampling distances, crucial for robust performance in this domain.
  2. Debiased Pseudo-Labeling: A significant challenge in SSL is the imbalance present not only in the original training data but also in the pseudo-labels generated by the model. These biased pseudo-labels can lead to skewed model predictions. To address this, the paper incorporates the DebiasPL method. This technique uses an adaptive debiasing module to adjust the logits of unlabeled instances and a marginal loss to enforce clearer class boundaries, especially for imbalanced classes. They also apply logit adjustment to mitigate biases in the original training labels.

Datasets

The primary dataset used for training and evaluation is xView. This is a large-scale, multi-spectral remote sensing dataset containing 8-band channels. For their experiments, the authors select a subset of 10 classes from the original 60, focusing on common objects like Storage Tanks, Helicopters, and Trucks. They utilize RGB bands from the original eight bands to form RGB images.

Key Insights and Ideas

  • Augmentations Matter (A Lot) for Remote Sensing: Generic data augmentation strategies that work well for natural images can be detrimental to remote sensing data. The paper highlights that careful, domain-specific augmentation design is crucial for effective SSL in this field.
  • Pseudo-Label Bias is a Real Problem: Machine-generated pseudo-labels, especially in imbalanced datasets, can introduce significant bias, hindering model performance. Addressing this bias is as important as leveraging the unlabeled data itself.
  • SSL Can Be Highly Data-Efficient: The proposed approach demonstrates that even with a small percentage of labeled data (e.g., 30%), substantial performance gains can be achieved, making SSL a powerful tool for data-scarce remote sensing applications.
  • Debiasing Techniques Transfer: The success of debiasing methods developed for natural images, like DebiasPL, can be effectively adapted and applied to the unique challenges of remote sensing data.

Key Results

The paper reports significant improvements on the xView dataset:

  • Using 30% of labeled annotations, their semi-supervised approach achieved a 7.1% accuracy gain over a supervised learning baseline.
  • It also outperformed the supervised state-of-the-art CDS method by 2.1%, even though CDS was trained with 100% labeled data.
  • In an initial setup with 10% labeled data, their approach achieved 80.8% accuracy, outperforming FixMatch by 4.6% and the supervised baseline by 5.5%.
  • Ablation studies showed that their adapted augmentations improved performance by 5.2% compared to using original DebiasPL augmentations.

Limitations and Future Work

  • Augmentation Sensitivity: The paper notes that the effectiveness of augmentations can vary, and their chosen set is empirically selected. Further exploration into optimal augmentation strategies for different remote sensing tasks might be beneficial.
  • Generalizability: While tested on xView, the authors' approach's performance on other remote sensing datasets with different characteristics (e.g., varying resolutions, sensor types, or class distributions) would be an important next step.
  • Computational Cost: While not explicitly detailed as a limitation, the use of multiple GPUs and extensive training suggests that computational resources are a consideration. Future work could explore more efficient training strategies.

Summary updated

Abstract

Deep learning has had remarkable success at analyzing handheld imagery such as consumer photos due to the availability of large-scale human annotations (e.g., ImageNet). However, remote sensing data lacks such extensive annotation and thus potential for supervised learning. To address this, we propose a highly effective semi-supervised approach tailored specifically to remote sensing data. Our approach encompasses two key contributions. First, we adapt the FixMatch framework to remote sensing data by designing robust strong and weak augmentations suitable for this domain. Second, we develop an effective semi-supervised learning method by removing bias in imbalanced training data resulting from both actual labels and pseudo-labels predicted by the model. Our simple semi-supervised framework was validated by extensive experimentation. Using 30\% of labeled annotations, it delivers a 7.1\% accuracy gain over the supervised learning baseline and a 2.1\% gain over the supervised state-of-the-art CDS method on the remote sensing xView dataset.
BibTeX
@misc{yeh2023debiased,
  author = {Chun-Hsiao Yeh and Xudong Wang and Stella X. Yu and Charles Hill and Zackery Steck and Scott Kangas and Aaron A. Reite},
  title = {{Debiased Learning for Remote Sensing Data}},
  year = {2023},
  eprint = {2312.15393},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2312.15393},
}
Figure