Label Matching Semi-Supervised Object Detection

Binbin Chen, Weijie Chen, Shicai Yang, Yunyi Xuan, Jie Song, Di Xie, Shiliang Pu, Mingli Song, Yueting Zhuang

Abstract

Semi-supervised object detection (SSOD) leverages a small amount of labeled data alongside a large pool of unlabeled images to train robust detectors. Current "mean teacher" approaches often suffer from label mismatch at both the distribution and instance levels. We introduce LabelMatch, a framework designed to rectify these issues using Adaptive label-distribution-aware Confidence Thresholds (ACT) and a novel Proposal Self-Assignment (PSA) mechanism. LabelMatch establishes a new state-of-the-art, achieving 25.81 mAP on MS-COCO with only 1% labeled data, significantly outperforming existing methods.

Introduction

The primary challenge in semi-supervised object detection is the effective utilization of unlabeled data. Most existing frameworks employ a teacher model to generate pseudo-labels for a student model. However, this process is prone to two types of label mismatch:

  1. Distribution-level mismatch: Fixed confidence thresholds often fail to account for class imbalances, leading to a biased pseudo-label distribution where some classes are over-represented by false positives and others are under-represented.
  2. Instance-level mismatch: Standard IoU-based label assignment is sensitive to the noise inherent in pseudo-labels. This can lead to "confirmation bias," where the student model overfits to the teacher's localization errors or produces ambiguous classification scores.
The two levels of label mismatch: distribution-level (class bias) and instance-level (assignment ambiguity).

Methodology

LabelMatch addresses these mismatches through three integrated components:

Re-distribution Mean Teacher (ACT)

To mitigate distribution-level mismatch, LabelMatch replaces fixed thresholds with Adaptive label-distribution-aware Confidence Thresholds (ACT). By assuming that labeled and unlabeled data share a similar underlying distribution, ACT adaptively calculates per-category thresholds. This ensures that the pseudo-label distribution remains aligned with the ground truth distribution of the labeled set.

Proposal Self-Assignment (PSA)

To resolve instance-level assignment ambiguity, the framework introduces Proposal Self-Assignment (PSA). Instead of relying on IoU-based matching for uncertain pseudo-labels, student-generated proposals are fed directly into the teacher model. The teacher then provides soft labels for these specific proposals, ensuring stable and accurate supervision even when pseudo-labels are noisy.

Reliable Pseudo Label Mining (RPLM)

LabelMatch categorizes pseudo-labels into "reliable" (high confidence) and "uncertain" (medium confidence). While reliable labels follow standard training, uncertain labels are processed via PSA. The Reliable Pseudo Label Mining (RPLM) strategy identifies high-quality labels within the uncertain pool based on prediction consistency (mean IoU and score) across adjacent boxes, gradually "upgrading" them as training progresses.

Overview of the LabelMatch framework, highlighting ACT, PSA, and RPLM components.

Experimental Results

LabelMatch was evaluated on MS-COCO and PASCAL-VOC benchmarks, demonstrating superior performance across various labeled data regimes:

  • MS-COCO (1% labeled): LabelMatch achieved 25.81 mAP, a 5.06-point improvement over the previous state-of-the-art (Unbiased Teacher).
  • MS-COCO (10% labeled): The model reached 35.49 mAP, significantly exceeding the supervised baseline of 23.86 mAP.
  • PASCAL-VOC: The framework achieved 85.48 AP50, compared to the supervised baseline of 72.63.

Qualitative analysis indicates that LabelMatch is particularly effective at reducing false positives in crowded scenes and detecting small objects that are typically missed by fixed-threshold methods.

Conclusion

LabelMatch effectively addresses the dual challenges of distribution and instance-level label mismatch in SSOD. By aligning pseudo-label distributions and refining instance-level assignments, it provides a more stable and accurate supervisory signal. While the framework currently assumes identical distributions between labeled and unlabeled data, its success highlights the importance of distribution alignment and robust assignment mechanisms in semi-supervised learning.

Summary updated

Abstract

Semi-supervised object detection has made significant progress with the development of mean teacher driven self-training. Despite the promising results, the label mismatch problem is not yet fully explored in the previous works, leading to severe confirmation bias during self-training. In this paper, we delve into this problem and propose a simple yet effective LabelMatch framework from two different yet complementary perspectives, i.e., distribution-level and instance-level. For the former one, it is reasonable to approximate the class distribution of the unlabeled data from that of the labeled data according to Monte Carlo Sampling. Guided by this weakly supervision cue, we introduce a re-distribution mean teacher, which leverages adaptive label-distribution-aware confidence thresholds to generate unbiased pseudo labels to drive student learning. For the latter one, there exists an overlooked label assignment ambiguity problem across teacher-student models. To remedy this issue, we present a novel label assignment mechanism for self-training framework, namely proposal self-assignment, which injects the proposals from student into teacher and generates accurate pseudo labels to match each proposal in the student model accordingly. Experiments on both MS-COCO and PASCAL-VOC datasets demonstrate the considerable superiority of our proposed framework to other state-of-the-arts. Code will be available at https://github.com/HIK-LAB/SSOD.
BibTeX
@inproceedings{chen2022label,
  author = {Binbin Chen and Weijie Chen and Shicai Yang and Yunyi Xuan and Jie Song and Di Xie and Shiliang Pu and Mingli Song and Yueting Zhuang},
  title = {{Label Matching Semi-Supervised Object Detection}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2022},
}
Figure