Abstract
Weakly Supervised Object Detection (WSOD) suffers from a significant performance gap compared to fully supervised methods, largely due to its inability to leverage modern architectural advancements. We present Salvage of Supervision (SoS), a three-stage framework that bridges this gap by iteratively refining pseudo-labels and employing semi-supervised learning. SoS enables the use of modern backbones like ResNet and Feature Pyramid Networks (FPN), achieving state-of-the-art results on PASCAL VOC and MS-COCO while significantly improving inference speed.
Introduction
Object detection typically requires "fully supervised" data, where every object is meticulously outlined with a bounding box. Producing such datasets is labor-intensive and expensive. Weakly Supervised Object Detection (WSOD) attempts to learn from "weak" labels—simple image-level tags indicating the presence of an object without spatial information.
However, WSOD has historically struggled to integrate modern detection advancements like ResNet backbones, FPN, and RoIAlign. These techniques, which have propelled Fully Supervised Object Detection (FSOD) to new heights, often degrade performance in traditional WSOD frameworks. Consequently, WSOD models remain slower and less accurate. This paper introduces the SoS framework to "salvage" supervisory signals from weak labels, allowing WSOD models to benefit from modern FSOD architectures.
Methods
The SoS-WSOD framework operates in three distinct stages to refine supervision iteratively:
- Improved WSOD (Stage 1): The process begins with a baseline WSOD model (based on OICR) enhanced with optimized proposal mining rules. By balancing precision and recall during proposal selection, the model ensures a more comprehensive initial set of object candidates.
- Pseudo-FSOD (Stage 2): The Stage 1 model generates pseudo-bounding boxes for the training set. To mitigate noise, the authors introduce Pseudo Groundtruth Filtering (PGF), which removes redundant, tiny, or low-confidence boxes. These salvaged labels are then used to train a modern FSOD detector (e.g., Faster R-CNN with ResNet50 and FPN) from scratch.
- Semi-Supervised Learning (Stage 3): To address remaining noise in pseudo-labels, the dataset is split into "clean" and "noisy" subsets based on a "small loss" criterion. The noisy subset is treated as unlabeled data within a semi-supervised framework (Unbiased Teacher) to further enhance model robustness.
Key Insights
- The Modernity Gap: Modern FSOD tools like RoIAlign can actually harm traditional WSOD because these models lack the initial strength to handle high-resolution features without better supervision.
- Filtering vs. Excavation: Rather than complex "excavation" of larger boxes, a robust filtering approach (PGF) proves more effective, especially on complex datasets like MS-COCO.
- Loss as Quality Proxy: Training loss serves as an effective proxy for pseudo-label quality; low-loss images are likely to have cleaner labels, while high-loss images are better utilized as unlabeled data in a semi-supervised context.
Results
SoS-WSOD significantly outperforms previous state-of-the-art methods across standard benchmarks:
- PASCAL VOC 2007/2012: Achieved 64.4 mAP₅₀ and 61.9 mAP₅₀ respectively, representing a substantial improvement over previous bests (~56.8).
- MS-COCO: Reached 32.8 mAP₅₀, nearly doubling the performance of many earlier WSOD methods on this challenging dataset.
- Efficiency: By utilizing a standard Faster R-CNN for the final stage, inference is approximately 3x faster than traditional WSOD models that rely on external object proposal generators.
Discussion and Limitations
While SoS-WSOD narrows the gap, a performance disparity remains compared to true full supervision (which exceeds 80 mAP on VOC). Current limitations include "part domination" (identifying only a portion of an object, like a head), difficulty distinguishing individual instances in crowded clusters, and occasional missing instances during pseudo-labeling. Future research will focus on more sophisticated data splitting and advanced semi-supervised techniques to further recover lost supervision.
Summary updated