Abstract
Training high-performance object detectors typically requires vast amounts of human-annotated data, which is both costly and time-consuming to produce. This paper introduces a semi-supervised learning (SSL) framework that leverages unlabeled data by generating high-quality pseudo-labels refined through co-occurrence matrix analysis. By capturing the statistical relationships between object classes—such as the high probability of a "keyboard" appearing near a "mouse"—the system filters out contextually inconsistent predictions. Evaluation on the MS-COCO dataset shows that this method consistently improves mean Average Precision (mAP) for popular architectures like Deformable ConvNets, SSD, and YOLO v2 without increasing inference-time complexity.
Introduction: The Data Bottleneck in Object Detection
Modern object detection models, particularly those based on Convolutional Neural Networks (CNNs), are data-hungry. While raw imagery is abundant, the manual process of drawing bounding boxes and assigning class labels is a significant bottleneck. Semi-supervised learning (SSL) offers a potential solution by using a small labeled dataset to bootstrap the labeling of a larger unlabeled pool. However, standard pseudo-labeling often suffers from "confirmation bias," where the model's initial errors are reinforced during subsequent training phases. To address this, we propose a method that uses external contextual information to validate and re-align these inferred labels.
Proposed Methodology
The proposed framework operates in two distinct phases, integrating a novel co-occurrence filtering mechanism to ensure the reliability of pseudo-labels.
Phase 1: Initial Training and Inference
The process begins by training a baseline detector (e.g., Deformable ConvNets) on a standard annotated dataset like COCO train17 + val17. This model is then used to perform inference on an unlabeled dataset (unlabeled17).
Phase 2: Co-occurrence Matrix Analysis and Re-training
The core innovation lies in how the raw predictions from Phase 1 are processed. Instead of accepting all high-confidence predictions, the system applies two layers of filtering:
- Confidence Thresholding: Only predictions exceeding a threshold $\rho$ are considered.
- Co-occurrence Filtering: A co-occurrence matrix is constructed from the labeled data, representing the conditional probability of one object class appearing given the presence of another. This matrix is used to re-scale the confidence of predicted labels. If a prediction (e.g., a "boat") appears in a context where it is statistically unlikely (e.g., a "living room"), its confidence score is penalized.
Finally, a new model is trained from scratch using the union of the original human-annotated data and the refined pseudo-labeled data.
Experimental Results
The effectiveness of the co-occurrence-based SSL method was evaluated using the MS-COCO dataset, with final benchmarking on the test-dev17 set. The method proved to be architecture-agnostic, providing consistent gains across different detector types:
| Model | Baseline mAP | SSL mAP | Improvement |
|---|---|---|---|
| D-ConvNets | 36.3 | 37.8 | +1.5 |
| SSD | 24.1 | 25.3 | +1.2 |
| YOLO v2 | 24.0 | 25.1 | +1.1 |
Qualitative analysis indicated that the co-occurrence prior successfully suppressed "hallucinated" objects that were contextually inappropriate while helping the model recover missed detections in complex, multi-object scenes.
Discussion and Conclusion
While the method significantly boosts detection accuracy, it does come with an increased training time due to the expanded dataset size. Furthermore, the final performance remains tied to the quality of the initial Phase 1 detector. Despite these limitations, the use of a co-occurrence matrix provides a robust, "bolt-on" strategy for improving object detectors without modifying their internal architecture or increasing computational overhead during deployment. Future work will explore more sophisticated relational modeling beyond simple pair-wise co-occurrence.
Summary updated