Discriminative Middle-Level Parts Mining for Object Detection

Dong Li, Yali Li, Shengjin WangIEICE Trans. Information and Systems 2015 · OpenReview

Abstract

Middle-level parts have attracted great attention in the computer vision community, acting as discriminative elements for objects. In this paper we propose an unsupervised approach to mine discriminative parts for object detection. This work features three aspects. First, we introduce an unsupervised, exemplar-based training process for part detection. We generate initial parts by selective search and then train part detectors by exemplar SVM. Second, a part selection model based on consistency and distinctiveness is constructed to select effective parts from the candidate pool. Third, we combine discriminative part mining with the deformable part model (DPM) for object detection. The proposed method is evaluated on the PASCAL VOC2007 and VOC2010 datasets. The experimental results demonstrate the effectiveness of our method for object detection.

Introduction

Object detection remains a fundamental challenge in computer vision due to significant intra-class variations caused by changes in viewpoint, lighting, occlusion, and non-rigid deformations. Traditional approaches like the Deformable Part Model (DPM) represent objects as a collection of parts. However, DPM often relies on suboptimal "greedy search" for part initialization, which can capture background noise rather than meaningful object components. Furthermore, manual part labeling is impractical for large-scale datasets.

This paper introduces an unsupervised framework to automatically mine "middle-level" parts—visually distinctive elements such as a bicycle's wheel or a bird's head—to enhance object detection. By leveraging selective search and exemplar SVMs within a robust selection framework, the proposed method identifies parts that are both consistent across instances and distinctive from the background.

Methodology

The proposed pipeline for discriminative part mining consists of three primary stages: initial part generation, part detector training, and discriminative part selection.

Overview of the proposed discriminative part mining pipeline, showing initial generation, training, and selection.

1. Initial Parts Generation

Instead of an exhaustive search, the method utilizes Selective Search to generate high-quality region proposals. A pre-trained "root model" identifies top-scoring object instances, and promising regions within these instances are selected as "seeds" for potential parts.

2. Part Detectors Training

For each candidate part, an Exemplar SVM (E-SVM) is trained. Unlike standard SVMs that require large labeled datasets, an E-SVM uses a single positive instance (the seed part) and a large set of negative background samples. This results in a detector specifically tuned to that unique visual pattern.

3. Discriminative Parts Selection

To ensure the selected parts are effective for detection, candidates are filtered based on two key metrics:

  • Consistency: Measures how frequently and reliably a part appears across different instances of the same object category.
  • Distinctiveness: Measures the detector's ability to trigger on the target object while remaining silent on background regions.
Visual comparison of 'best' and 'worst' parts based on consistency and distinctiveness criteria.

These scores are fused to rank candidates, and the top-performing parts are integrated into the DPM framework. A spatial voting mechanism is then used to initialize part locations more accurately than traditional greedy methods.

Experimental Results

The method was rigorously evaluated on the PASCAL VOC 2007 and VOC 2010 benchmarks, covering 20 diverse object categories.

  • PASCAL VOC 2007: The proposed method achieved a Mean Average Precision (mAP) of 35.0%, outperforming the standard DPM baseline of 32.5%.
  • PASCAL VOC 2010: The method maintained its lead with an mAP of 30.1%, compared to the 28.6% achieved by DPM.
  • Category Performance: The approach proved exceptionally effective for rigid objects, surpassing the DPM baseline in 18 out of 20 categories.
  • Efficiency: The system is computationally efficient, with part detector training taking approximately 1 second and inference averaging 2 seconds per image on standard hardware.

Discussion and Future Work

A key insight from this research is the differing importance of selection criteria across object types. Distinctiveness is paramount for rigid objects (e.g., cars) with highly specific parts. In contrast, consistency is more critical for deformable objects (e.g., animals), where parts must be recognizable despite shape changes.

While the unsupervised mining of parts provides a significant boost over manual or greedy initialization, limitations remain. The current reliance on HOG (Histogram of Oriented Gradients) features suggests that integrating CNN-based features could further enhance performance. Additionally, the model still faces challenges with severe occlusion and categories with high internal variance. Future research will explore iterative learning schemes to jointly optimize object components and part detectors.

Summary updated

Abstract

Middle-level parts have attracted great attention in the computer vision community, acting as discriminative elements for objects. In this paper we propose an unsupervised approach to mine discriminative parts for object detection. This work features three aspects. First, we introduce an unsupervised, exemplar-based training process for part detection. We generate initial parts by selective search and then train part detectors by exemplar SVM. Second, a part selection model based on consistency and distinctiveness is constructed to select effective parts from the candidate pool. Third, we combine discriminative part mining with the deformable part model (DPM) for object detection. The proposed method is evaluated on the PASCAL VOC2007 and VOC2010 datasets. The experimental results demons-trate the effectiveness of our method for object detection.
BibTeX
@article{li2015discriminative,
  author = {Dong Li and Yali Li and Shengjin Wang},
  title = {{Discriminative Middle-Level Parts Mining for Object Detection}},
  journal = {IEICE Trans. Information and Systems},
  year = {2015},
  url = {https://openreview.net/forum?id=c1cd64xGsL},
}
Figure