RS-SSKD: Self-Supervision Equipped with Knowledge Distillation for Few-Shot Remote Sensing Scene Classification

Pei Zhang, Ying Li, Dong Wang, Jiyue Wang2021 · DOI

TL;DR

This paper introduces RS-SSKD, a novel method for few-shot remote sensing scene classification that leverages self-supervised learning and knowledge distillation to create powerful image representations. By focusing on learning discriminative embeddings, RS-SSKD achieves state-of-the-art performance on challenging remote sensing datasets, addressing the critical need for accurate classification with limited labeled data.

What Problem Does This Paper Address?

Remote sensing (RS) scene classification is crucial for applications like urban planning and environmental monitoring. While the volume of RS imagery is rapidly increasing, obtaining large, meticulously labeled datasets for training deep learning models is a significant bottleneck. This is particularly challenging for "few-shot learning" (FSL) scenarios, where models must generalize to new categories with only a handful of examples. Traditional FSL methods often rely on complex meta-learning architectures, but this paper argues that the quality of the learned feature representations is paramount. The core problem is how to train models that can effectively learn from very limited data in the RS domain, especially when dealing with the inherent complexities and variations in these images.

Methods

The proposed RS-SSKD method comprises two main modules: a self-supervised knowledge distillation (SSKD) embedding module and a meta-learning module.

  • SSKD Embedding Module: This module is designed to learn robust and discriminative feature embeddings. It employs a novel two-branch network that takes pairs of original and transformed images (e.g., flipped, scaled) as input. Crucially, it incorporates Class Activation Maps (CAMs) to guide the network in identifying and focusing on the most relevant category-specific regions within an image, thereby reducing distractions from irrelevant parts. A self-supervised loss is defined based on the consistency of CAMs between original and transformed images. This module is trained using a combined loss function that includes both the self-supervised loss and a standard cross-entropy loss.
  • Self-Distillation: To further enhance performance and prevent overfitting, a round of self-distillation is applied. Here, the trained self-supervised network is cloned into a teacher and a student model. The student model is trained to mimic the teacher's predictions, using a combination of cross-entropy loss and Kullback-Leibler divergence loss between their outputs.
  • Meta-Learning Module: The powerful embeddings generated by the SSKD module are then fed into a standard meta-learning framework, specifically Prototypical Networks (ProtoNets). This module is trained using an episodic approach, learning to classify new categories based on a few support examples.

Datasets

The proposed method was evaluated on two widely used remote sensing scene classification datasets:

  • NWPU-RESISC45: This dataset contains 45 categories with 700 images each, totaling 31,500 images of size 256x256 pixels.
  • RSD46-WHU: This dataset comprises 46 categories with a larger number of images per category (500-3000), totaling 117,000 images.

The datasets were split into meta-training (SEEN categories), meta-validation (HELD-OUT categories), and meta-test (UNSEEN categories) sets, following established protocols in few-shot learning research.

Key Insights and Ideas

  • Focus on Representation Learning: The paper strongly advocates that high-quality feature embeddings are more critical for few-shot learning performance than complex meta-learning architectures alone. This shifts the focus from intricate meta-learning algorithms to the foundational task of learning discriminative representations.
  • CAMs for Region Focus: The innovative use of Class Activation Maps (CAMs) within a self-supervised framework is a key insight. By forcing the network to align attention maps across transformed images, it learns to pinpoint semantically relevant regions, which is particularly beneficial for the complex and varied nature of remote sensing imagery.
  • Self-Supervision for Attention Consistency: The self-supervised task of enforcing attention consistency across transformed image pairs is a clever way to leverage unlabeled data to improve feature discriminability without requiring explicit human annotations for the self-supervised task itself.
  • Self-Distillation for Regularization: Applying self-distillation, even for a single round, proves effective in further refining the learned representations and mitigating overfitting, which is a common issue in low-data regimes.

Key Results

RS-SSKD achieved state-of-the-art (SOTA) performance on both the NWPU-RESISC45 and RSD46-WHU datasets for both 1-shot and 5-shot classification scenarios.

  • On NWPU-RESISC45, RS-SSKD achieved an accuracy of 70.64% ± 0.22% for 1-shot and 86.26% ± 0.12% for 5-shot.
  • On RSD46-WHU, RS-SSKD achieved 71.73% ± 0.25% for 1-shot and 85.90% ± 0.15% for 5-shot.

These results consistently surpassed previous methods, including those that also utilized the ResNet-12 backbone. Qualitative results showed that the CAMs generated by RS-SSKD effectively highlighted the most relevant regions for classification, demonstrating the method's ability to capture discriminative features. The training time was comparable to other SOTA methods, with a modest increase due to the SSKD pre-training stage.

Limitations and Future Work

  • Meta-Learning Procedure: While the paper emphasizes the importance of embeddings, it still relies on a standard meta-learning procedure for the final classification. The authors suggest future work could involve improving the meta-learning process itself, perhaps by learning network backbones.
  • Real-World Applications: The authors acknowledge that while the experimental results are encouraging, they are not yet at the level of human-level performance. They propose investigating more real-world applications of their method.
  • Training Time: Although comparable to other SOTA methods, the SSKD module does add to the overall training time. Further optimization of the self-supervised and distillation steps could be explored.

Summary updated

BibTeX
@misc{zhang2021rs,
  author = {Pei Zhang and Ying Li and Dong Wang and Jiyue Wang},
  title = {{RS-SSKD: Self-Supervision Equipped with Knowledge Distillation for Few-Shot Remote Sensing Scene Classification}},
  year = {2021},
  doi = {10.3390/s21051566},
}
Figure