CoDeL: A Human Co-detection and Labeling Framework

Jianping Shi, Renjie Liao, Jiaya JiaICCV 2013 · OpenReview

Abstract

We propose a co-detection and labeling (CoDeL) framework to identify persons that contain self-consistent appearance in multiple images. Our CoDeL model builds upon the deformable part-based model to detect human hypotheses and exploits cross-image correspondence via a matching classifier. Relying on a Gaussian process, this matching classifier models the similarity of two hypotheses and efficiently captures the relative importance contributed by various visual features, reducing the adverse effect of scattered occlusion. Further, the detector and matching classifier together make our model fit into a semi-supervised co-training framework, which can get enhanced results with a small amount of labeled training data. Our CoDeL model achieves decent performance on existing and new benchmark datasets.

Introduction

Traditional human detection and identification systems often operate on a single-image basis or rely heavily on face recognition. However, in real-world photo collections, such as family albums or TV show episodes, faces are frequently occluded, turned away, or blurred. This paper addresses the human co-detection problem: given a set of images containing a group of people, the goal is to detect every person and label them by identity.

Example of human co-detection and labeling on the HCD dataset, where identities are tracked across varying poses.

The core challenge lies in the fact that while poses and backgrounds vary significantly, a person's clothing and overall appearance usually remain consistent within a specific event. CoDeL exploits this "self-consistent appearance" as a powerful prior to improve detection accuracy in scenarios where traditional methods fail.

The CoDeL Framework

The CoDeL framework is built on three main pillars designed to bridge the gap between detection and identity matching.

1. Human Representation

The model utilizes a modified Deformable Part-based Model (DPM), representing a person as a "root" filter for the whole body and several flexible "part" filters (head, torso, etc.). A dedicated face filter is added to provide a strong anchor when facial features are visible.

2. Gaussian Process (GP) Matching Classifier

To determine if two detections represent the same person, a GP classifier models similarity based on color histograms, texture (LBP), and SIFT features across different body parts. The GP is particularly effective because it learns the relative importance of different features, making it robust to partial occlusions. For instance, if a person is sitting behind a table, the GP learns to down-weight the importance of the legs for identification.

3. Semi-Supervised Co-Training

This iterative "engine" allows the model to bootstrap itself using a small amount of labeled data.

The CoDeL co-training framework showing the iterative update loop between the detector and the matching classifier.
  • Detector → Matcher: High-confidence detections are used as "weakly labeled" samples to retrain the matching classifier.
  • Matcher → Detector: High-confidence matches across images provide evidence to "rescue" weak detections, which are then used to retrain the detector.

Experimental Evaluation

The authors evaluated CoDeL on a standard Pedestrian Dataset and a new Human Co-Detection (HCD) Dataset. The HCD dataset features diverse poses (sitting, eating) and complex backgrounds, providing a realistic test for general co-detection.

Key Results

  • Detection Performance: On the HCD dataset, CoDeL achieved an Average Precision (AP) of 74.94%, significantly outperforming the standard DPM (69.94%). On the Pedestrian dataset, the improvement was even more pronounced, jumping from 59.7% to 74.4%.
  • Matching Accuracy: The GP framework reached 91.43% matching accuracy on the pedestrian set by combining multiple visual features.
  • Iterative Improvement: Detection performance steadily increased over 15 rounds of co-training, validating the effectiveness of the semi-supervised approach.

Conclusion and Future Work

CoDeL demonstrates that cross-image appearance consistency can significantly boost human detection and labeling. However, the system can still be challenged by individuals wearing similar uniforms or extreme changes in clothing. Future work includes integrating deeper spatial relationships into the matching process and developing an online version of the algorithm for real-time processing.

Summary updated

Abstract

We propose a co-detection and labeling (CoDeL) framework to identify persons that contain self-consistent appearance in multiple images. Our CoDeL model builds upon the deformable part-based model to detect human hypotheses and exploits cross-image correspondence via a matching classifier. Relying on a Gaussian process, this matching classifier models the similarity of two hypotheses and efficiently captures the relative importance contributed by various visual features, reducing the adverse effect of scattered occlusion. Further, the detector and matching classifier together make our model fit into a semi-supervised co-training framework, which can get enhanced results with a small amount of labeled training data. Our CoDeL model achieves decent performance on existing and new benchmark datasets.
BibTeX
@inproceedings{shi2013codel,
  author = {Jianping Shi and Renjie Liao and Jiaya Jia},
  title = {{CoDeL: A Human Co-detection and Labeling Framework}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2013},
  url = {https://openreview.net/forum?id=ry4rmWfu-S},
}
Figure