Abstract
Recent development of object detection mainly depends on deep learning with large-scale benchmarks. However, collecting such fully-annotated data is often difficult or expensive for real-world applications, which restricts the power of deep neural networks in practice. Alternatively, humans can detect new objects with little annotation burden, since humans often use the prior knowledge to identify new objects with few elaborately-annotated examples, and subsequently generalize this capacity by exploiting objects from wild images. Inspired by this procedure of learning to detect, we propose a novel Progressive Object Transfer Detection (POTD) framework. Specifically, we make three main contributions in this paper. First, POTD can leverage various object supervision of different domains effectively into a progressive detection procedure. Via such human-like learning, one can boost a target detection task with few annotations. Second, POTD consists of two delicate transfer stages, i.e., Low-Shot Transfer Detection (LSTD), and Weakly-Supervised Transfer Detection (WSTD). In LSTD, we distill the implicit object knowledge of source detector to enhance target detector with few annotations. It can effectively warm up WSTD later on. In WSTD, we design a recurrent object labelling mechanism for learning to annotate weakly-labeled images. More importantly, we exploit the reliable object supervision from LSTD, which can further enhance the robustness of target detector in the WSTD stage. Finally, we perform extensive experiments on a number of challenging detection benchmarks with different settings. The results demonstrate that, our POTD outperforms the recent state-of-the-art approaches.
Introduction
Deep learning has revolutionized object detection, but its success is typically tethered to massive, fully-annotated datasets where every object is meticulously boxed. In real-world scenarios, labeling thousands of bounding boxes is often prohibitively expensive and slow. Existing alternatives like Weakly-Supervised Detection (WSD) and Low-Shot Learning face significant challenges: WSD often struggles with precise localization, while Low-Shot Learning frequently suffers from severe overfitting due to the scarcity of data.
The Progressive Object Transfer Detection (POTD) framework addresses these issues by mimicking human learning. Humans use prior knowledge to identify new objects from a few examples and then refine that understanding by observing the object in unannotated contexts. POTD combines low-shot and weakly-supervised learning into a single, progressive pipeline to bridge the gap between limited supervision and high performance.
Methodology
The POTD framework is built on a Faster-RCNN-style architecture with an SSD-style Region Proposal Network (RPN). It operates in two primary stages:
1. Low-Shot Transfer Detection (LSTD)
This "warm-up" phase involves fine-tuning a detector pre-trained on a large source domain (e.g., MS COCO) using a target domain with as few as one annotation per class. To mitigate overfitting, two regularization terms are introduced: Source Detection Knowledge (SDK), which distills "objectness" knowledge from the source model, and Background Depression (BD), which penalizes high activations in background regions to focus the model on the few available labeled objects.
2. Weakly-Supervised Transfer Detection (WSTD)
Following the warm-up, the model generalizes using a large pool of images with only image-level labels. A key component is Recurrent Object Labelling (ROL), which iteratively mines confident object proposals and selectively labels contextual background. This mechanism provides a cleaner training signal compared to traditional weakly-supervised methods that often mislabel redundant proposals.
Experiments and Results
The authors evaluated POTD across several challenging transfer tasks using source domains like MS COCO and ImageNet2015, and target domains including VOC2007, VOC2010, and VOC2012.
Key results include:
- Performance: On VOC2007, POTD with 30 shots plus weakly-labeled data achieved 71.3 mAP, outperforming standard fully-supervised Faster R-CNN (69.9 mAP).
- Efficiency: With only 1 fully-annotated shot per class, POTD reached 62.6 mAP, significantly higher than pure weakly-supervised methods which typically hover around 40-50 mAP.
- Robustness: The use of an SSD-style RPN, where the regressor is shared across classes, proved more robust in low-shot scenarios than standard class-specific RPNs.
Conclusion
POTD demonstrates that a small amount of full supervision can provide a critical foundation for weakly-supervised learning, preventing the model from collapsing into local optima. By progressively transferring knowledge, the framework achieves performance rivaling fully-supervised detectors with a fraction of the labeling effort. Future work could explore its effectiveness across more disparate domains and optimize the computational overhead of the recurrent labeling process.
Summary updated