Abstract
Weakly supervised object detection (WSOD) using only image-level annotations has attracted growing attention over the past few years. Existing approaches using multiple instance learning easily fall into local optima, because such mechanism tends to learn from the most discriminative object in an image for each category. Therefore, these methods suffer from missing object instances which degrade the performance of WSOD. To address this problem, this paper introduces an end-to-end object instance mining (OIM) framework for weakly supervised object detection. OIM attempts to detect all possible object instances existing in each image by introducing information propagation on the spatial and appearance graphs, without any additional annotations. During the iterative learning process, the less discriminative object instances from the same class can be gradually detected and utilized for training. In addition, we design an object instance reweighted loss to learn larger portion of each object instance to further improve the performance. The experimental results on two publicly available databases, VOC 2007 and 2012, demonstrate the efficacy of proposed approach.
Introduction
Object detection typically requires fully supervised data with precise bounding-box annotations, which are expensive to produce. Weakly Supervised Object Detection (WSOD) aims to train detectors using only image-level labels. However, WSOD faces two primary challenges: the "Part Problem," where models focus on the most discriminative part of an object rather than the whole body, and the "Missing Instance Problem," where standard Multiple Instance Learning (MIL) algorithms often detect only the single most confident instance, ignoring others.
The proposed Object Instance Mining (OIM) framework addresses these issues by "mining" missing instances and encouraging the model to capture the full extent of each object.
The OIM Framework
The OIM framework integrates two novel components into a standard detection pipeline to improve instance discovery and localization.
Object Instance Mining via Graphs
OIM utilizes spatial and appearance-based graphs to propagate information from high-confidence "seed" instances to hidden ones:
- Spatial Graphs: These identify proposals with high spatial overlap with the core instance.
- Appearance Graphs: These leverage feature vectors to find regions that are visually similar to the core instance, even if they are spatially distant.
Instance Reweighted Loss
To mitigate the tendency to focus on small discriminative parts, an Instance Reweighted (IR) loss is introduced. This loss assigns higher weights to surrounding, less-discriminative proposals relative to the core proposal, forcing the network to pay more attention to the rest of the object's body during training.
Experimental Evaluation
The OIM framework was evaluated on the PASCAL VOC 2007 and 2012 benchmarks.
Evolution of Learning
A key strength of OIM is its ability to gradually discover more complete object representations during training. As shown in the visualizations, the model's heatmaps evolve from isolated discriminative points to comprehensive maps covering multiple instances and full object bodies.
Quantitative Performance
On VOC 2007, the OIM+IR method achieved a state-of-the-art 50.1% mAP, significantly outperforming previous methods like OICR (41.2%) and PCL (43.5%). Localization accuracy (CorLoc) also improved to 67.2%, demonstrating that the mined boxes align well with actual object boundaries. The method is particularly effective for categories that frequently appear in groups, such as sheep.
Conclusion and Future Work
OIM significantly advances WSOD by effectively mining multiple instances and fuller object bodies through graph propagation and reweighted loss. While it sets new benchmarks, challenges remain in handling highly non-rigid objects (e.g., humans) and separating instances in very crowded scenes. Future research will focus on refining spatial relationships to better handle these complex scenarios.
Summary updated