Abstract
Self-supervised learning (SSL) has revolutionized feature representation for image classification, yet its benefits for object detection are often bottlenecked by the localization task. Traditional SSL pre-trains the model backbone, leaving detection-specific modules like the Region Proposal Network (RPN) to be initialized randomly. This paper introduces ADePT (Aligned DEtection Pre-Training), a method that pre-trains the RPN using unsupervised region proposals as pseudo-labels. ADePT effectively aligns pre-training with the downstream detection task, significantly reducing localization errors. Our approach achieves state-of-the-art results in label-scarce and few-shot scenarios across MS COCO, SODA10M, and PASCAL VOC benchmarks.
Introduction
While self-supervised pre-training has become a standard for image classification, object detection presents a unique challenge: it requires both identifying "what" an object is (classification) and "where" it is (localization). Current SSL methods for detection, such as SoCo or DetCo, focus primarily on the feature extractor (backbone). Consequently, when fine-tuning on small datasets, the randomly initialized Region Proposal Network (RPN) struggles to learn accurate localization. This "task misalignment" makes localization error the primary performance bottleneck in label-scarce settings.
Methods: Aligned DEtection Pre-Training (ADePT)
ADePT addresses task misalignment by introducing a pretext task specifically for the RPN. The core components include:
- Unsupervised Pseudo-Ground Truth: The method leverages Selective Search, a classic heuristic algorithm, to generate candidate bounding boxes for unlabeled images.
- RPN Pre-training: The RPN is trained to regress its predictions toward these Selective Search boxes. While noisy, these pseudo-labels provide a foundational understanding of "objectness" and geometry.
- Contrastive Framework: ADePT is integrated into a BYOL-style framework, matching object-level features across multiple augmented views of an image.
- Flexible Strategies: The authors evaluate ADePT (separate), which pre-trains the RPN on a fixed SSL backbone, and ADePT (joint), which trains the backbone and detector modules simultaneously.
Key Insights: Localization as the Bottleneck
Error analysis using the TIDE toolbox confirms that localization error is the dominant factor limiting the performance of SSL-based detectors. By pre-training the RPN, ADePT directly targets this weakness.
The success of ADePT suggests that even simple heuristics like Selective Search are sufficient to initialize modern neural networks for complex spatial tasks. This highlights the importance of architectural alignment—pre-training the entire detector architecture, not just the backbone.
Experimental Results
ADePT consistently outperforms previous state-of-the-art methods across diverse benchmarks:
- Label-Scarce COCO: With only 1% of labeled data, ADePT (separate) achieves 15.3 AP, surpassing SoCo. Qualitative analysis shows improved confidence and better localization of small objects.
- Autonomous Driving (SODA10M): ADePT (joint) reached 43.6 AP, a substantial improvement over the supervised baseline (32.9 AP) and competing SSL methods.
- Few-Shot Detection: On PASCAL VOC (K=1), ADePT achieved 2.14 mAP, nearly quadrupling the performance of SoCo (0.56 mAP).
Limitations and Future Work
Despite its success, ADePT relies on Selective Search, which is computationally slow and produces noisy proposals. Future iterations could benefit from more advanced unsupervised proposal generators. Additionally, while ADePT performs well generally, it can be sensitive to domain shifts if the pre-training data (ImageNet) lacks the specific environmental conditions of the downstream task (e.g., extreme lighting). Future work will explore extending this alignment strategy to instance segmentation using unsupervised mask labels.
Summary updated