TL;DR
This paper introduces "Salvage of Supervision" (SoS), a novel framework that significantly boosts Weakly Supervised Object Detection (WSOD) performance by cleverly leveraging all available supervisory signals: image-level labels, generated pseudo-labels, and semi-supervised learning. By bridging the gap between WSOD and modern Fully Supervised Object Detection (FSOD) techniques, SoS achieves state-of-the-art results while also enabling faster inference.
What Problem Does This Paper Address?
Object detection, a crucial task in computer vision, typically relies on extensive datasets with precise bounding box annotations for each object. However, obtaining such detailed labels is costly and time-consuming. Weakly Supervised Object Detection (WSOD) aims to overcome this by training models using only image-level labels (e.g., knowing a picture contains a "cat" but not where the cat is). While this reduces annotation effort, WSOD methods have historically lagged far behind their fully supervised counterparts (FSOD) in accuracy. A key challenge is that modern, highly effective FSOD techniques (like advanced backbones and detection architectures) often perform poorly when directly applied to WSOD due to the inherent noise and ambiguity of image-level supervision. This paper tackles this performance gap by proposing a method to "salvage" and effectively utilize all available supervisory signals to improve WSOD.
Methods
The proposed "Salvage of Supervision" (SoS) framework operates in three stages:
- Improved WSOD (Stage 1): This stage trains an initial WSOD detector. The authors propose an improved version of the OICR method, incorporating refined proposal mining rules and a multi-input strategy to strike a better balance between proposal recall and precision. This stage aims to produce the best possible initial detection results using only image-level labels.
- Pseudo-FSOD (Stage 2): Here, the goal is to leverage modern FSOD techniques. The WSOD detector from Stage 1 generates "pseudo" bounding box annotations. A novel Pseudo Groundtruth Filtering (PGF) algorithm is introduced to refine these noisy pseudo-labels, making them higher quality than previous methods. This filtered set of pseudo-labels then allows for training a fully supervised detector using state-of-the-art FSOD architectures (e.g., ResNet backbones, FPN, RoIAlign).
- Semi-Supervised Detection (SSOD) (Stage 3): Recognizing that even filtered pseudo-labels can be noisy, this stage treats the pseudo-labeled dataset as a noisy dataset. It splits the data into a "clean" subset (images with high-confidence pseudo-labels) and an "unlabeled" subset (images with noisier pseudo-labels). A semi-supervised learning approach, inspired by methods like Unbiased Teacher, is then applied to further improve the detector by learning from both the clean labeled data and the unlabeled data. A key innovation here is a novel splitting criterion for object detection that uses a "small loss" idea adapted for the task.
Datasets
The authors evaluate their SoS-WSOD framework on three standard object detection benchmarks:
- PASCAL VOC2007
- PASCAL VOC2012
- MS-COCO
Key Insights and Ideas
- Harnessing all signals: The core idea is that WSOD shouldn't be limited to just image-level labels. By actively salvaging and integrating pseudo-labels and semi-supervised learning, significant performance gains are achievable.
- Bridging the FSOD-WSOD gap: Modern FSOD techniques are powerful but brittle when applied directly to noisy WSOD data. SoS provides a robust pathway to integrate these techniques by first cleaning up the supervision signal.
- PGF for better pseudo-labels: Simple filtering of detection results is crucial. The PGF algorithm effectively removes low-confidence and redundant proposals, leading to higher quality pseudo-groundtruth for the FSOD stage.
- Noisy label handling is key: Treating the pseudo-labeled dataset as noisy and employing a careful splitting strategy for semi-supervised learning is vital for further improvement, preventing the model from overfitting to incorrect labels.
- Compatibility: The SoS framework is designed to be compatible with various existing WSOD methods and can readily adopt modern FSOD backbones and architectures.
Key Results
- State-of-the-art performance: SoS-WSOD achieves significant improvements over existing WSOD methods. On VOC2007, it reaches 64.4 mAP50 (with ResNet50 backbone), on VOC2012, 61.9 mAP50, and on MS-COCO, 16.6 mAP50:95.
- Stage-wise improvements: Each stage of the SoS framework contributes to performance gains. Stage 2 (pseudo-FSOD) provides a substantial boost, and Stage 3 (SSOD) further refines the results.
- Faster inference: By leveraging modern FSOD architectures and eliminating the need for external proposal generation (like Selective Search), SoS-WSOD achieves faster inference speeds compared to traditional WSOD methods.
- Effective PGF: The PGF algorithm significantly outperforms simpler pseudo-labeling methods like "Top-One" and the W2F method, especially on more complex datasets like MS-COCO.
Limitations and Future Work
- Performance gap remains: Despite significant improvements, SoS-WSOD still exhibits a performance gap compared to fully supervised detection, particularly on challenging datasets like MS-COCO.
- WSOD challenges persist: The framework still faces common WSOD issues such as part domination, missing instances, and clustered instances.
- Future directions: The authors suggest exploring ways to salvage even more supervisory signals, developing better methods for splitting noisy datasets, and investigating more powerful semi-supervised detection techniques for WSOD. They also plan to continue addressing the fundamental challenges of WSOD.
Summary updated