Cross-Scene Classification of Remote Sensing Images Based on General-Specific Prototype Contrastive Learning

Puhua Chen, Yu Qiu, Lei Guo, Xiangrong Zhang, Fang Liu, Licheng Jiao, Lingling Li

Abstract

Cross-scene classification in remote sensing (RS) faces significant challenges due to domain gaps and class imbalances. We present General-Specific Prototype Contrastive Learning (GSPCL), a framework that leverages category-general and category-specific features for robust domain adaptation. By introducing a Bidirectional Weighted Prototype (BWP) strategy and utilizing low-confidence samples through contrastive learning, GSPCL effectively addresses intra-class diversity and the "long-tail" problem. Experimental results across seven major datasets demonstrate that GSPCL significantly outperforms state-of-the-art methods, particularly in imbalanced scenarios.

Introduction

Remote sensing imagery classification often suffers from the "domain gap"—where models trained on a source region fail in a target region due to variations in environmental conditions and sensor characteristics. Two primary issues hinder effective cross-scene classification: intra-class diversity (objects in the same class looking different) and class imbalance (the "long-tail" problem). Most existing domain adaptation methods struggle when target data is imbalanced, often neglecting rare classes. GSPCL addresses these by moving away from simple feature alignment toward a nuanced prototype-based approach.

Methodology

The GSPCL framework introduces several innovations to bridge the domain gap by distinguishing between "category-general" features (low-level traits like color and texture) and "category-specific" features (high-level semantic information).

The GSPCL framework architecture, illustrating the shared CNN backbones, the use of general and specific prototypes, and the Judge module for processing low-confidence features.

Bidirectional Weighted Prototype (BWP) Strategy

To ensure that rare classes are adequately represented, the BWP strategy combines confidence-based selection with class-balanced sampling. Unlike standard clustering or Argmax methods, BWP generates robust prototypes even for classes with few samples, mitigating bias toward dominant categories and ensuring even rare classes have representative centers in the feature space.

Comparison of prototype generation strategies: Argmax, Top-2, and the proposed Bidirectional Weighted Prototype (BWP) which ensures better class coverage for imbalanced data.

Low-Confidence Contrastive Learning

Rather than discarding low-confidence samples as noise, GSPCL utilizes them as "anchors" in a contrastive learning setup. By forcing the model to align these difficult boundary-case samples with known source-domain prototypes, the model develops sharper decision boundaries and resolves semantic ambiguities that high-confidence samples might miss.

Experimental Results

The model was evaluated across three settings using seven optical RS datasets (AID, UCM, WHU-RS, RSD46-WHU, RSSCN7, NWPU-RESISC45, and PatternNet):

  • Setting 1 (Standard Cross-Scene): GSPCL achieved 92.36% average accuracy, a 5.0% improvement over the second-best method (AMRAN).
  • Setting 2 (Large Domain Gap): Reached 87.9% average accuracy, with near-perfect performance in specific tasks like R→P (99.9%).
  • Setting 3 (Long-Tail/Imbalanced): In a rigorous class-imbalanced scenario where source and target distributions were reversed, GSPCL maintained 75.1% accuracy, while competing methods saw performance drops exceeding 20%.

Conclusion

GSPCL offers a robust solution for cross-scene RS classification by treating low-confidence data as an asset and ensuring balanced prototype representation. Despite its complexity, it maintains a competitive parameter count. Future work will focus on making prototype construction more adaptive and automated, reducing the need for manual hyperparameter tuning.

Summary updated

Abstract

In recent years, constrained by the challenges associated with expensive data annotation and poor generalization ability in supervised models, domain adaptation has been proposed to effectively mitigate domain gaps, which has gained significant attention in the field of remote sensing. However, most existing methods do not consider the inherent characteristics of remote sensing images when formulating adaptive strategies. In addition, there has been limited research on addressing class-imbalanced data situations, leading to undesirable performance in domain adaptation tasks involving long-tailed datasets. To overcome the aforementioned limitations, based on the analysis of intraclass diversity and intradomain style differences of remote sensing images, we propose a novel prototype contrastive learning framework called general-specific prototype contrastive learning (GSPCL). Due to the unreliability of clustering samples in conventional prototype-based clustering methods, the bidirectional weighted prototype strategy is proposed to optimize this loophole. Consequently, more robust prototypes are constructed in both domains, serving as mediators to reduce domain discrepancies bidirectionally. Particularly, often overlooked in most methods, we incorporate low-confidence sample features into the contrastive learning process alongside these prototypes to further guide the model to address feature alignment and long-tail issues effectively. Finally, in order to verify the superiority of our proposed method, we adhere to two existing experimental settings and construct an extra optical remote sensing domain adaptation dataset with class-imbalanced scenarios. In the first two experimental settings, GSPCL outperforms the second-ranked approach by 5.0% and 4.0% in average accuracy. Furthermore, our approach exhibits highly competitive results in handling long-tailed data scenarios.
BibTeX
@article{chen2024cross,
  author = {Puhua Chen and Yu Qiu and Lei Guo and Xiangrong Zhang and Fang Liu and Licheng Jiao and Lingling Li},
  title = {{Cross-Scene Classification of Remote Sensing Images Based on General-Specific Prototype Contrastive Learning}},
  journal = {IEEE J. Sel. Top. Appl. Earth Obs. Remote. Sens.},
  year = {2024},
  doi = {10.1109/jstars.2024.3379437},
}
Figure