Abstract
We propose a novel way of using videos to obtain high precision object proposals for weakly-supervised object detection. Existing weakly-supervised detection approaches use off-the-shelf proposal methods like edge boxes or selective search to obtain candidate boxes. These methods provide high recall but at the expense of thousands of noisy proposals. Thus, the entire burden of finding the few relevant object regions is left to the ensuing object mining step. To mitigate this issue, we focus instead on improving the precision of the initial candidate object proposals. Since we cannot rely on localization annotations, we turn to video and leverage motion cues to automatically estimate the extent of objects to train a Weakly-supervised Region Proposal Network (W-RPN). We use the W-RPN to generate high precision object proposals, which are in turn used to re-rank high recall proposals like edge boxes or selective search according to their spatial overlap. Our W-RPN proposals lead to significant improvement in performance for state-of-the-art weakly-supervised object detection approaches on PASCAL VOC 2007 and 2012.
Introduction
Object detection typically requires thousands of expensive, manually-drawn bounding box annotations. Weakly-supervised object detection (WSOD) attempts to bypass this by training models using only image-level labels (e.g., "this image contains a car" but not "the car is at these coordinates").
The standard pipeline for WSOD starts by generating thousands of "candidate boxes" using off-the-shelf methods like Selective Search or Edge Boxes. This creates a "needle in a haystack" problem: the model must find the one or two correct boxes among 2,000+ noisy candidates without ever seeing a ground-truth box. Consequently, models often get stuck in local optima, focusing only on the most "distinctive" part of an object (like a bird's beak) or including bits of the background that always appear with the object (like a boat in water).
Methodology: Leveraging Motion for Precision
The authors propose a three-stage approach to move from "high recall/low precision" proposals to "high precision" ones by leveraging unlabeled video data.
1. Learning from Motion
In videos, objects that move differently from their background provide a "free" supervisory signal. The authors use unsupervised motion segmentation (Non-Local Consensus Voting) on large video datasets to identify moving objects. They then treat the tightest boxes around these segments as "pseudo ground-truth."
2. The Weakly-supervised Region Proposal Network (W-RPN)
A Region Proposal Network (W-RPN) is trained on these pseudo-labels. To ensure quality, an image classifier selects video frames where the object is most salient, and an "outlier score" filters out noisy or blurry motion masks. This ensures the network learns from the most reliable examples.
3. High-Precision Proposals & Ranking
When applied to static images, the W-RPN generates a small set (~10) of very reliable proposals. These are combined with Class Activation Maps (CAMs) to ensure they overlap with the relevant object class. Finally, these high-precision boxes are used to re-rank the thousands of noisy standard proposals using a novel ranking loss, forcing the detector to prefer boxes that align with the W-RPN's predictions.
Key Insights and Results
Precision over Recall
Traditional proposal methods prioritize recall (making sure the object is in one of the 2,000 boxes). This paper argues that for weak supervision, precision is more important. Having 10 "mostly right" boxes is better for a weak learner than 2,000 "mostly wrong" ones.
Motion as a Proxy for Extent
One of the biggest failures in WSOD is "part-based detection" (detecting just a face instead of a body). Because an entire object usually moves as a single unit in a video, motion cues naturally define the full extent of the object, providing the spatial information that image-level labels lack.
Performance Benchmarks
The method was evaluated on the PASCAL VOC 2007 and 2012 benchmarks, using video data from YouTube-Bounding Boxes and YFCC100m for training.
- Performance Boost: Adding W-RPN proposals to the standard WSDDN (Weakly Supervised Deep Detection Network) increased Mean Average Precision (mAP) on PASCAL VOC 2007 from 34.8% to 39.4%.
- State-of-the-Art: When combined with more recent architectures like OICR, the method achieved 46.9% mAP, outperforming previous methods that relied on complex "track and transfer" schemes.
Conclusion
By shifting the focus from high-recall to high-precision proposals, this work demonstrates that unlabeled video can provide a powerful supervisory signal for object detection. The W-RPN successfully captures the full extent of objects, mitigating the common WSOD issue of focusing on discriminative parts. While the method relies on motion, camera movement often provides sufficient cues even for stationary objects, making it a robust approach for improving weakly-supervised learning.
Summary updated