C-MIDN: Coupled Multiple Instance Detection Network With Segmentation Guidance for Weakly Supervised Object Detection

Gao Yan, Boxiao Liu, Nan Guo, Xiaochun Ye, Fang Wan, Haihang You, Dongrui Fan

Abstract

Weakly Supervised Object Detection (WSOD) aims to train object detectors using only image-level labels, bypassing the need for expensive bounding box annotations. However, standard Multiple Instance Learning (MIL) approaches often suffer from the "discriminative part" problem, where models focus on the most recognizable features (e.g., a cat's face) rather than the entire object. We present the Coupled Multiple Instance Detection Network (C-MIDN), which employs two coupled networks to iteratively discover complementary object regions. By integrating Segmentation Guided Proposal Removal (SGPR), the system forces the model to look beyond initial detections while ensuring the preservation of essential object information. C-MIDN achieves state-of-the-art results on PASCAL VOC 2007, 2012, and MS-COCO benchmarks.

Introduction

Object detection typically requires "strong supervision," involving thousands of manually annotated images. WSOD offers a more scalable alternative by utilizing image-level labels (e.g., "cat," "dog"). The prevailing framework for WSOD is Multiple Instance Learning (MIL), which treats an image as a bag of proposals. Unfortunately, MIL models are often "lazy," converging on the most discriminative parts that satisfy the training loss. This results in bounding boxes that are either too tight around specific parts or too loose and inaccurate.

Methodology

The C-MIDN architecture introduces two primary innovations to address the part-focus problem:

Coupled MIDNs and Proposal Removal

The system utilizes two detection networks in sequence. Once the first network identifies the most discriminative part of an object, those proposals are removed from the input of the second network. This adversarial cooperation forces the second network to identify remaining object parts or other instances in the scene.

Overview of the C-MIDN framework showing the coupling of two MIDN modules guided by SGPR.

Segmentation Guided Proposal Removal (SGPR)

To prevent the second network from losing all positive instances—which could happen if the first network correctly identified the entire object—the authors introduce SGPR. Using rough, weakly-supervised segmentation maps as a spatial "budget," SGPR evaluates the coverage of the first network's detections. If a detection covers most of the segmented area, proposals are kept; if it only covers a small portion, they are removed to encourage further discovery.

Online Detector Refinement (ODR)

The outputs from the coupled networks are integrated and refined through an iterative process that progressively tightens bounding boxes, improving localization accuracy.

Experimental Results

C-MIDN was evaluated on the PASCAL VOC 2007, 2012, and MS-COCO benchmarks.

Quantitative Performance

On PASCAL VOC 2007, C-MIDN achieved a mean Average Precision (mAP) of 53.6%, significantly outperforming previous state-of-the-art methods such as W2F (52.4%) and PCL (48.8%). Similar gains were observed on PASCAL VOC 2012, where the model reached 50.3% mAP.

Qualitative Analysis

Visual results demonstrate that C-MIDN successfully expands detections from isolated parts to full object bodies. In cluttered scenes, the model effectively distinguishes between multiple distinct objects that baseline models often group together.

Qualitative comparison showing how C-MIDN expands detections from discriminative parts to full object extents compared to baseline MIDN.

Conclusion

C-MIDN addresses the fundamental limitations of MIL-based weakly supervised detection by forcing the model to explore complementary object regions. While highly effective for animals and rigid objects, the model still faces challenges with highly articulated classes, such as humans, where part-based focus remains persistent. Future research will focus on end-to-end integration of segmentation and detection to further enhance robustness.

Summary updated

Abstract

Weakly supervised object detection (WSOD) that only needs image-level annotations has obtained much attention recently. By combining convolutional neural network with multiple instance learning method, Multiple Instance Detection Network (MIDN) has become the most popular method to address the WSOD problem and been adopted as the initial model in many works. We argue that MIDN inclines to converge to the most discriminative object parts, which limits the performance of methods based on it. In this paper, we propose a novel Coupled Multiple Instance Detection Network (C-MIDN) to address this problem. Specifically, we use a pair of MIDNs, which work in a complementary manner with proposal removal. The localization information of the MIDNs is further coupled to obtain tighter bounding boxes and localize multiple objects. We also introduce a Segmentation Guided Proposal Removal (SGPR) algorithm to guarantee the MIL constraint after the removal and ensure the robustness of C-MIDN. Through a simple implementation of the C-MIDN with online detector refinement, we obtain 53.6% and 50.3% mAP on the challenging PASCAL VOC 2007 and 2012 benchmarks respectively, which significantly outperform the previous state-of-the-arts.
BibTeX
@inproceedings{yan2019c,
  author = {Gao Yan and Boxiao Liu and Nan Guo and Xiaochun Ye and Fang Wan and Haihang You and Dongrui Fan},
  title = {{C-MIDN: Coupled Multiple Instance Detection Network With Segmentation Guidance for Weakly Supervised Object Detection}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2019},
  doi = {10.1109/iccv.2019.00993},
}
Figure