Object Instance Mining for Weakly Supervised Object Detection

Chenhao Lin, Siwen Wang, Dongqi Xu, Yu Lu, Wayne Zhang

Abstract

Weakly supervised object detection (WSOD) using only image-level annotations has attracted growing attention over the past few years. Existing approaches using multiple instance learning easily fall into local optima, because such mechanism tends to learn from the most discriminative object in an image for each category. Therefore, these methods suffer from missing object instances which degrade the performance of WSOD. To address this problem, this paper introduces an end-to-end object instance mining (OIM) framework for weakly supervised object detection. OIM attempts to detect all possible object instances existing in each image by introducing information propagation on the spatial and appearance graphs, without any additional annotations. During the iterative learning process, the less discriminative object instances from the same class can be gradually detected and utilized for training. In addition, we design an object instance reweighted loss to learn larger portion of each object instance to further improve the performance. The experimental results on two publicly available databases, VOC 2007 and 2012, demonstrate the efficacy of proposed approach.

Introduction

Training robust object detectors typically requires extensive bounding-box annotations, which are expensive and time-consuming to collect. Weakly Supervised Object Detection (WSOD) offers a compelling alternative by learning from image-level labels alone. However, traditional WSOD methods based on Multiple Instance Learning (MIL) often suffer from two critical issues:

  1. Missing Instances: Models tend to focus on the single most discriminative instance in an image, ignoring other valid objects of the same class.
  2. Part-based Detection: Models often converge on highly distinct object parts (e.g., a bird's head) rather than the entire object body.

This paper proposes the Object Instance Mining (OIM) framework to overcome these limitations by iteratively discovering all object instances through spatial and appearance relationships.

Methodology

The OIM Framework Architecture

The proposed OIM framework is an end-to-end pipeline that builds upon standard detection architectures like WSDDN. It introduces a dedicated Object Instance Mining module alongside the Multiple Instance Detection (MID) branch.

The OIM architecture integrates a backbone network with ROI pooling, followed by Multiple Instance Detection (MID) and Object Instance Mining (OIM) modules.

Information Propagation via Graphs

The core of OIM is the use of two graph types to propagate labels from high-confidence "core instances" to other potential proposals:

  • Spatial Graph ($G^s$): Identifies proposals with high spatial overlap with the core instance, helping the model capture the full extent of an object.
  • Appearance Graph ($G^a$): Measures similarity in feature space. If a proposal looks like the core instance, it is mined as a new instance, regardless of its location in the image.
Graph-based mining: (a) initial core instance, (b-c) propagation via spatial and appearance graphs to discover additional instances, (d) final multi-instance detection.

Object Instance Reweighted Loss ($L_{OIR}$)

To prevent the model from collapsing onto the most discriminative parts, the authors introduce a reweighted loss. This loss assigns weights to proposals based on their relationship to the core instance, forcing the network to learn from a larger portion of the object and improving localization accuracy.

Experiments and Results

The framework was evaluated on the PASCAL VOC 2007 and 2012 datasets.

  • PASCAL VOC 2007: OIM achieved a mean Average Precision (mAP) of 50.1%, significantly outperforming previous state-of-the-art methods like OICR (41.2%) and PCL (43.5%).
  • Localization Accuracy: Using the CorLoc metric, OIM reached 67.2%, demonstrating superior ability in wrapping the entire object.
  • Pseudo-Ground Truth: When used to train a standard Fast-RCNN, the mined boxes yielded a performance of 52.6% mAP, validating the high quality of the discovered instances.

Conclusion

Object Instance Mining effectively addresses the "local optima" problem in weakly supervised detection. By leveraging spatial and appearance graphs, the model can discover multiple instances and more complete object extents without additional supervision. While challenges remain with highly non-rigid objects and background clutter, OIM represents a significant step toward bridging the gap between weakly and fully supervised object detection.

Summary updated

Abstract

Weakly supervised object detection (WSOD) using only image-level annotations has attracted growing attention over the past few years. Existing approaches using multiple instance learning easily fall into local optima, because such mechanism tends to learn from the most discriminative object in an image for each category. Therefore, these methods suffer from missing object instances which degrade the performance of WSOD. To address this problem, this paper introduces an end-to-end object instance mining (OIM) framework for weakly supervised object detection. OIM attempts to detect all possible object instances existing in each image by introducing information propagation on the spatial and appearance graphs, without any additional annotations. During the iterative learning process, the less discriminative object instances from the same class can be gradually detected and utilized for training. In addition, we design an object instance reweighted loss to learn larger portion of each object instance to further improve the performance. The experimental results on two publicly available databases, VOC 2007 and 2012, demonstrate the efficacy of proposed approach.
BibTeX
@inproceedings{lin2020object,
  author = {Chenhao Lin and Siwen Wang and Dongqi Xu and Yu Lu and Wayne Zhang},
  title = {{Object Instance Mining for Weakly Supervised Object Detection}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2020},
  url = {https://openreview.net/forum?id=DIGqaOA7xJj},
}
Figure