Low-Confidence Samples Mining for Semi-supervised Object Detection

Guandu Liu, Fangyuan Zhang, Tianxiang Pan, Bin Wang

Abstract

Reliable pseudo-labels from unlabeled data are fundamental to semi-supervised object detection (SSOD). Current state-of-the-art methods typically employ high confidence thresholds to filter pseudo-labels, which inadvertently discards many valid predictions and limits model recall. This paper introduces Low-confidence Samples Mining (LSM), a novel method to efficiently utilize low-confidence pseudo-labels. LSM incorporates a Pseudo Information Mining (PIM) branch that operates on low-resolution feature maps to extract reliable large-area instances. Furthermore, a Self-Distillation (SD) module is employed to facilitate mutual learning between the PIM and the main detection branch. Experimental results on the MS-COCO benchmark demonstrate that LSM achieves a 3.54% mAP improvement over existing methods under a 5% labeling ratio, proving its effectiveness across both CNN-based and Transformer-based detectors.

Introduction

Semi-supervised object detection (SSOD) seeks to leverage large volumes of unlabeled data to augment a small set of labeled examples. The dominant paradigm in this field is the "Mean Teacher" framework, where a teacher model generates pseudo-labels on unlabeled images to supervise a student model. To maintain the quality of these pseudo-labels, researchers usually apply a high confidence threshold (e.g., 0.7 or 0.9).

While this "high-threshold" strategy ensures high precision, it suffers from a significant drawback: low recall. Many valid objects are predicted with moderate confidence and are consequently discarded. This paper identifies a critical observation: the reliability of a prediction is often correlated with the object's scale. Large objects, even when predicted with lower confidence, tend to have higher Intersection-over-Union (IoU) with ground truth compared to small objects at the same confidence level. LSM is designed to exploit this scale-dependent reliability to mine informative samples that traditional methods ignore.

The LSM Framework

The Low-confidence Samples Mining (LSM) framework introduces two key components to the standard detector architecture:

1. Pseudo Information Mining (PIM)

The PIM branch is an auxiliary detection head that processes a downsampled version of the input image (typically reducing resolution by half). The intuition behind PIM is that low-resolution feature maps naturally emphasize global structures and large-scale objects while suppressing small-scale noise. By applying a lower filtering threshold ($\alpha$) specifically within this branch, the model can identify large-area instances that are structurally sound but might not meet the strict confidence requirements of the main branch. These "mined" samples provide additional supervision that enhances the detector's coverage.

2. Self-Distillation (SD)

To integrate the knowledge from the PIM branch back into the main detector, the authors propose a Self-Distillation (SD) mechanism. This module calculates the KL divergence between the classification probability distributions of the main branch and the PIM branch. This mutual learning approach allows the main branch to benefit from the "soft" labels generated by the PIM branch, effectively transferring the reliability of low-confidence large objects into the primary detection pipeline.

LSM is designed to be architecture-agnostic. It can be integrated into two-stage detectors like Faster R-CNN or end-to-end transformer architectures like Deformable-DETR, making it a versatile tool for various SSOD tasks.

Experimental Evaluation

The effectiveness of LSM was validated through extensive experiments on standard benchmarks:

  • MS-COCO Performance: On the MS-COCO dataset with only 5% labeled data, LSM achieved 31.97% mAP, representing a +3.54% improvement over the UBTeacher baseline. Similar gains were observed at 1% and 10% labeling ratios.
  • Cross-Domain Robustness: In a challenging COCO-to-ImageNet transfer task, where the unlabeled data comes from a different distribution, LSM improved Deformable-DETR performance by 1.3% mAP. This suggests that the PIM branch effectively filters out out-of-distribution noise.
  • Generalization: The method also showed consistent improvements on the PASCAL VOC dataset, confirming that the benefits of low-confidence mining are not dataset-specific.

Discussion and Insights

The success of LSM stems from two primary insights:

  1. Scale-Aware Reliability: The correlation between object size and pseudo-label quality is a powerful prior. By acknowledging that a low-confidence "bus" is more likely to be real than a low-confidence "bird," the model can safely lower its guard for specific object classes.
  2. Structural Filtering: Using low resolution as a proxy for structural importance allows the model to ignore pixel-level noise that often plagues low-confidence predictions in high-resolution maps.

Conclusion

Low-confidence Samples Mining (LSM) addresses a fundamental bottleneck in semi-supervised object detection: the loss of information due to over-cautious pseudo-labeling. By introducing a scale-aware mining branch and a self-distillation strategy, LSM significantly improves model recall without sacrificing precision. The substantial gains on MS-COCO and the method's compatibility with various architectures establish LSM as a robust framework for practical SSOD applications. Future work may focus on extending these mining techniques to small objects and further optimizing the computational efficiency of the multi-branch architecture.

Summary updated

Abstract

Reliable pseudo-labels from unlabeled data play a key role in semi-supervised object detection (SSOD). However, the state-of-the-art SSOD methods all rely on pseudo-labels with high confidence, which ignore valuable pseudo-labels with lower confidence. Additionally, the insufficient excavation for unlabeled data results in an excessively low recall rate thus hurting the network training. In this paper, we propose a novel Low-confidence Samples Mining (LSM) method to utilize low-confidence pseudo-labels efficiently. Specifically, we develop an additional pseudo information mining (PIM) branch on account of low-resolution feature maps to extract reliable large-area instances, the IoUs of which are higher than small-area ones. Owing to the complementary predictions between PIM and the main branch, we further design self-distillation (SD) to compensate for both in a mutually-learning manner. Meanwhile, the extensibility of the above approaches enables our LSM to apply to Faster-RCNN and Deformable-DETR respectively. On the MS-COCO benchmark, our method achieves 3.54% mAP improvement over state-of-the-art methods under 5% labeling ratios.
BibTeX
@inproceedings{liu2023low,
  author = {Guandu Liu and Fangyuan Zhang and Tianxiang Pan and Bin Wang},
  title = {{Low-Confidence Samples Mining for Semi-supervised Object Detection}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2023},
  url = {https://openreview.net/forum?id=WiyVDnN7Lb},
}
Figure