Region Proposal Network Pre-Training Helps Label-Efficient Object Detection

Linus Ericsson, Nanqing Dong, Yongxin Yang, Ales Leonardis, Steven McDonagh

Abstract

Self-supervised learning (SSL) has revolutionized feature representation for image classification, yet its benefits for object detection are often bottlenecked by the localization task. Traditional SSL pre-trains the model backbone, leaving detection-specific modules like the Region Proposal Network (RPN) to be initialized randomly. This paper introduces ADePT (Aligned DEtection Pre-Training), a method that pre-trains the RPN using unsupervised region proposals as pseudo-labels. ADePT effectively aligns pre-training with the downstream detection task, significantly reducing localization errors. Our approach achieves state-of-the-art results in label-scarce and few-shot scenarios across MS COCO, SODA10M, and PASCAL VOC benchmarks.

Introduction

While self-supervised pre-training has become a standard for image classification, object detection presents a unique challenge: it requires both identifying "what" an object is (classification) and "where" it is (localization). Current SSL methods for detection, such as SoCo or DetCo, focus primarily on the feature extractor (backbone). Consequently, when fine-tuning on small datasets, the randomly initialized Region Proposal Network (RPN) struggles to learn accurate localization. This "task misalignment" makes localization error the primary performance bottleneck in label-scarce settings.

Methods: Aligned DEtection Pre-Training (ADePT)

ADePT addresses task misalignment by introducing a pretext task specifically for the RPN. The core components include:

  1. Unsupervised Pseudo-Ground Truth: The method leverages Selective Search, a classic heuristic algorithm, to generate candidate bounding boxes for unlabeled images.
  2. RPN Pre-training: The RPN is trained to regress its predictions toward these Selective Search boxes. While noisy, these pseudo-labels provide a foundational understanding of "objectness" and geometry.
  3. Contrastive Framework: ADePT is integrated into a BYOL-style framework, matching object-level features across multiple augmented views of an image.
  4. Flexible Strategies: The authors evaluate ADePT (separate), which pre-trains the RPN on a fixed SSL backbone, and ADePT (joint), which trains the backbone and detector modules simultaneously.

Key Insights: Localization as the Bottleneck

Error analysis using the TIDE toolbox confirms that localization error is the dominant factor limiting the performance of SSL-based detectors. By pre-training the RPN, ADePT directly targets this weakness.

Error analysis comparing ADePT with other SSL methods, showing a significant reduction in localization error.

The success of ADePT suggests that even simple heuristics like Selective Search are sufficient to initialize modern neural networks for complex spatial tasks. This highlights the importance of architectural alignment—pre-training the entire detector architecture, not just the backbone.

Experimental Results

ADePT consistently outperforms previous state-of-the-art methods across diverse benchmarks:

  • Label-Scarce COCO: With only 1% of labeled data, ADePT (separate) achieves 15.3 AP, surpassing SoCo. Qualitative analysis shows improved confidence and better localization of small objects.
  • Autonomous Driving (SODA10M): ADePT (joint) reached 43.6 AP, a substantial improvement over the supervised baseline (32.9 AP) and competing SSL methods.
  • Few-Shot Detection: On PASCAL VOC (K=1), ADePT achieved 2.14 mAP, nearly quadrupling the performance of SoCo (0.56 mAP).
Qualitative comparison of object detection results across different SSL pre-training methods.

Limitations and Future Work

Despite its success, ADePT relies on Selective Search, which is computationally slow and produces noisy proposals. Future iterations could benefit from more advanced unsupervised proposal generators. Additionally, while ADePT performs well generally, it can be sensitive to domain shifts if the pre-training data (ImageNet) lacks the specific environmental conditions of the downstream task (e.g., extreme lighting). Future work will explore extending this alignment strategy to instance segmentation using unsupervised mask labels.

Summary updated

BibTeX
@article{ericsson2024region,
  author = {Linus Ericsson and Nanqing Dong and Yongxin Yang and Ales Leonardis and Steven McDonagh},
  title = {{Region Proposal Network Pre-Training Helps Label-Efficient Object Detection}},
  journal = {Neurocomputing},
  year = {2024},
  doi = {10.1016/j.neucom.2024.127376},
}
Figure