Co-Occurrence Matrix Analysis-Based Semi-Supervised Training for Object Detection

Min-Kook Choi, Jaehyeong Park, Jihun Jung, Heechul Jung, Jin-Hee Lee, Woong-Jae Won, Woo Young Jung, Jincheol Kim, Soon Kwon

Abstract

Training high-performance object detectors typically requires vast amounts of human-annotated data, which is both costly and time-consuming to produce. This paper introduces a semi-supervised learning (SSL) framework that leverages unlabeled data by generating high-quality pseudo-labels refined through co-occurrence matrix analysis. By capturing the statistical relationships between object classes—such as the high probability of a "keyboard" appearing near a "mouse"—the system filters out contextually inconsistent predictions. Evaluation on the MS-COCO dataset shows that this method consistently improves mean Average Precision (mAP) for popular architectures like Deformable ConvNets, SSD, and YOLO v2 without increasing inference-time complexity.

Introduction: The Data Bottleneck in Object Detection

Modern object detection models, particularly those based on Convolutional Neural Networks (CNNs), are data-hungry. While raw imagery is abundant, the manual process of drawing bounding boxes and assigning class labels is a significant bottleneck. Semi-supervised learning (SSL) offers a potential solution by using a small labeled dataset to bootstrap the labeling of a larger unlabeled pool. However, standard pseudo-labeling often suffers from "confirmation bias," where the model's initial errors are reinforced during subsequent training phases. To address this, we propose a method that uses external contextual information to validate and re-align these inferred labels.

Proposed Methodology

The proposed framework operates in two distinct phases, integrating a novel co-occurrence filtering mechanism to ensure the reliability of pseudo-labels.

Overview of the two-phase semi-supervised training pipeline.

Phase 1: Initial Training and Inference

The process begins by training a baseline detector (e.g., Deformable ConvNets) on a standard annotated dataset like COCO train17 + val17. This model is then used to perform inference on an unlabeled dataset (unlabeled17).

Phase 2: Co-occurrence Matrix Analysis and Re-training

The core innovation lies in how the raw predictions from Phase 1 are processed. Instead of accepting all high-confidence predictions, the system applies two layers of filtering:

  1. Confidence Thresholding: Only predictions exceeding a threshold $\rho$ are considered.
  2. Co-occurrence Filtering: A co-occurrence matrix is constructed from the labeled data, representing the conditional probability of one object class appearing given the presence of another. This matrix is used to re-scale the confidence of predicted labels. If a prediction (e.g., a "boat") appears in a context where it is statistically unlikely (e.g., a "living room"), its confidence score is penalized.
Visualization of co-occurrence matrices and normalization techniques used to refine pseudo-labels.

Finally, a new model is trained from scratch using the union of the original human-annotated data and the refined pseudo-labeled data.

Experimental Results

The effectiveness of the co-occurrence-based SSL method was evaluated using the MS-COCO dataset, with final benchmarking on the test-dev17 set. The method proved to be architecture-agnostic, providing consistent gains across different detector types:

Model Baseline mAP SSL mAP Improvement
D-ConvNets 36.3 37.8 +1.5
SSD 24.1 25.3 +1.2
YOLO v2 24.0 25.1 +1.1

Qualitative analysis indicated that the co-occurrence prior successfully suppressed "hallucinated" objects that were contextually inappropriate while helping the model recover missed detections in complex, multi-object scenes.

Discussion and Conclusion

While the method significantly boosts detection accuracy, it does come with an increased training time due to the expanded dataset size. Furthermore, the final performance remains tied to the quality of the initial Phase 1 detector. Despite these limitations, the use of a co-occurrence matrix provides a robust, "bolt-on" strategy for improving object detectors without modifying their internal architecture or increasing computational overhead during deployment. Future work will explore more sophisticated relational modeling beyond simple pair-wise co-occurrence.

Summary updated

BibTeX
@inproceedings{choi2018co,
  author = {Min-Kook Choi and Jaehyeong Park and Jihun Jung and Heechul Jung and Jin-Hee Lee and Woong-Jae Won and Woo Young Jung and Jincheol Kim and Soon Kwon},
  title = {{Co-Occurrence Matrix Analysis-Based Semi-Supervised Training for Object Detection}},
  booktitle = {ICIP},
  year = {2018},
  doi = {10.1109/icip.2018.8451360},
}
Figure