Generalization-Enhanced Few-Shot Object Detection in Remote Sensing

Hui Lin, Nan Li, Pengjuan Yao, Kexin Dong, Yuhan Guo, Danfeng Hong, Ying Zhang, Congcong WenIEEE TCSVT 2025 · DOI · OpenReview

TL;DR

This paper introduces GE-FSOD, a novel framework designed to significantly improve few-shot object detection (FSOD) in remote sensing imagery. By enhancing the feature extraction, region proposal, and classification stages with novel components like CFPAN, MRRPN, and GCL, GE-FSOD achieves state-of-the-art performance on challenging datasets, addressing the critical need for accurate object detection with limited labeled data in this domain.

What Problem Does This Paper Address?

Object detection in remote sensing imagery is notoriously difficult due to high resolution, multi-scale features, and diverse object characteristics. While deep learning has made strides, it typically requires vast amounts of labeled data. Acquiring such data, especially for rare or novel objects, is costly and time-consuming. Few-shot learning (FSL) offers a solution by enabling models to learn from minimal examples, but existing FSOD methods struggle with the unique complexities of remote sensing data, limiting their generalization capabilities. This paper tackles this challenge by proposing a GE-FSOD model specifically designed to enhance generalization in remote sensing FSOD tasks.

Methods

The GE-FSOD model builds upon the standard backbone-neck-head architecture but introduces several key innovations:

  • Cross-Level Fusion Pyramid Attention Network (CFPAN): This novel "neck" module replaces the traditional Feature Pyramid Network (FPN). CFPAN enhances multi-scale feature representation by integrating dual attention mechanisms (channel and spatial, inspired by CBAM) and cross-level feature fusion. This allows for adaptive weighting of semantic and spatial information, improving the detection of objects at various scales.
  • Multi-Stage Refinement Region Proposal Network (MRRPN): This "head" component replaces the standard Region Proposal Network (RPN). MRRPN employs a multi-stage refinement strategy to iteratively improve the accuracy and effectiveness of region proposals. It uses dilated convolutions for broader receptive fields and deformable convolutions in later stages to capture intricate geometric features, crucial for diverse object shapes.
  • Generalized Classification Loss (GCL): This novel classification loss function is designed to improve performance in few-shot scenarios. It incorporates "placeholder nodes" in the classification layer, reserved for future novel classes and regularized during base training to prevent learning base class features. During fine-tuning, these nodes are activated to adapt to novel classes while maintaining base class performance.

The model is trained in two stages: base training on abundant data for general feature learning, followed by fine-tuning on limited novel class data.

Datasets

The authors evaluated their GE-FSOD model on two prominent remote sensing object detection datasets:

  • DIOR (Dataset for Object Detection in Aerial Images): A large and diverse dataset with 23,463 images and over 192,000 annotated instances across 20 categories. It features significant variations in scale, quality, and environmental conditions.
  • NWPU VHR-10: A dataset comprising 715 high-resolution images from Google Earth and 85 from Vaihingen. It includes a "negative image set" and is divided into optimizing and testing sets with ten object classes.

Key Insights and Ideas

  • The Power of Adaptive Attention: The integration of attention mechanisms (CFPAN) and adaptive fusion strategies is crucial for handling the scale variations and complex features inherent in remote sensing imagery. Static fusion methods are insufficient.
  • Iterative Refinement for Proposals: A multi-stage refinement process for region proposals (MRRPN) is more effective than a single-stage approach, allowing for progressively more accurate localization, especially for challenging objects.
  • Proactive Novelty Handling: The GCL with placeholder nodes is a clever way to prepare the model for unseen classes during base training, preventing catastrophic forgetting and enabling better adaptation during fine-tuning.
  • Remote Sensing Specific Challenges: The paper highlights that standard FSOD techniques often fall short on remote sensing data, necessitating specialized architectural and loss function designs.

Key Results

The GE-FSOD model achieved state-of-the-art performance on both the DIOR and NWPU VHR-10 datasets across various few-shot settings (3-shot, 5-shot, 10-shot, and 20-shot).

  • DIOR Dataset: For example, in Split 1 (3-shot), GE-FSOD achieved 31.69% accuracy, surpassing the best baseline by 2.89%. In Split 3 (5-shot), it reached 22.58%, exceeding the baseline by 3.51%.
  • NWPU VHR-10 Dataset: The model consistently outperformed baselines, achieving 60.43% in the 3-shot setting and 86.10% in the 20-shot setting, with notable improvements over SAE-FSDet and G-FSDet in qualitative comparisons.
  • Ablation Studies: CFPAN improved accuracy by up to 4.41% (3-shot DIOR), CBAM by 1.98% (3-shot DIOR), and the MRRPN with 3 stages provided optimal performance. GCL also showed significant gains, increasing accuracy from 25.86% to 31.69% (3-shot DIOR) compared to standard cross-entropy loss.

Limitations and Future Work

  • Computational Cost: While achieving superior accuracy, GE-FSOD has higher computational complexity (FLOPs) and slower inference speed compared to some baseline methods. The authors acknowledge this trade-off, arguing that precision is paramount in remote sensing FSOD.
  • Future Work: The paper implicitly suggests that further optimization for computational efficiency while maintaining high accuracy would be beneficial. Exploring more advanced attention mechanisms or refinement strategies could also be avenues for future research. The effectiveness of the proposed components across a wider range of remote sensing datasets and object types could also be investigated.

Summary updated

Abstract

Remote sensing object detection is particularly challenging due to the high resolution, multi-scale features, and diverse ground object characteristics inherent in satellite and UAV imagery. These challenges necessitate more advanced approaches for effective object detection in such environments. While deep learning methods have achieved remarkable success in remote sensing object detection, they typically rely on large amounts of labeled data. Acquiring sufficient labeled data, particularly for novel or rare objects, is both challenging and time-consuming in remote sensing scenarios, limiting the generalization capabilities of existing models. To address these challenges, few-shot learning (FSL) has emerged as a promising approach, aiming to enable models to learn new classes from limited labeled examples. Building on this concept, few-shot object detection (FSOD) specifically targets object detection challenges in data-limited conditions. However, the generalization capability of FSOD models, particularly in remote sensing, is often constrained by the complex and diverse characteristics of the objects present in such environments. In this paper, we propose the Generalization-Enhanced Few-Shot Object Detection (GE-FSOD) model to improve the generalization capability in remote sensing FSOD tasks. Our model introduces three key innovations: the Cross-Level Fusion Pyramid Attention Network (CFPAN) for enhanced multi-scale feature representation, the Multi-Stage Refinement Region Proposal Network (MRRPN) for more accurate region proposals, and the Generalized Classification Loss (GCL) for improved classification performance in few-shot scenarios. Extensive experiments on the DIOR and NWPU VHR-10 datasets show that our model achieves state-of-the-art performance for few-shot object detection in remote sensing.
BibTeX
@article{lin2025generalization,
  author = {Hui Lin and Nan Li and Pengjuan Yao and Kexin Dong and Yuhan Guo and Danfeng Hong and Ying Zhang and Congcong Wen},
  title = {{Generalization-Enhanced Few-Shot Object Detection in Remote Sensing}},
  journal = {IEEE Trans. Circuits Syst. Video Technol.},
  year = {2025},
  doi = {10.1109/tcsvt.2025.3528262},
}
Figure