Low-Confidence Samples Mining for Semi-supervised Object Detection

Guandu Liu, Fangyuan Zhang, Tianxiang Pan, Jun-Hai Yong, Bin Wang

Abstract

Reliable pseudo labels from unlabeled data play a key role in semi-supervised object detection (SSOD). However, state-of-the-art SSOD methods typically rely on high-confidence pseudo labels, ignoring valuable lower-confidence samples. This insufficient excavation of unlabeled data results in low recall rates that hinder network training. We propose Low-confidence Samples Mining (LSM) to utilize these labels efficiently. By developing a Pseudo Information Mining (PIM) branch using low-resolution feature maps, we extract reliable large-area instances. We further design self-distillation (SD) for mutual learning between branches. LSM improves mAP by 3.54% over state-of-the-art methods on MS-COCO with a 5% labeling ratio.

Introduction

Semi-Supervised Object Detection (SSOD) leverages a small labeled dataset alongside a large unlabeled dataset. The standard "mean teacher" paradigm involves a teacher model generating pseudo-labels on unlabeled data to train a student model. To maintain quality, existing methods often discard predictions below a high confidence threshold (e.g., 0.9). However, this approach overlooks a significant amount of useful training data.

The core motivation for this work is the observation that confidence scores do not perfectly correlate with bounding box accuracy. Specifically, large bounding boxes with low confidence scores often exhibit high Intersection-over-Union (IoU) with ground truth, whereas small boxes with high confidence are frequently false positives. This scale-to-quality relationship suggests that current methods are leaving valuable data on the table.

Empirical evidence showing that larger pseudo-labels tend to have higher IoU quality, even at lower confidence levels.

Methodology

To exploit these low-confidence samples without introducing excessive noise, we propose the Low-confidence Samples Mining (LSM) framework, which operates alongside the standard detection branch.

Pseudo Information Mining (PIM)

Instead of globally lowering the confidence threshold, which would flood the model with noisy small-object detections, PIM utilizes a parallel branch with downsampled feature maps. Since small objects are naturally filtered out at lower resolutions, the network is forced to focus on large-area candidates. This allows for a safely lowered confidence threshold (e.g., 0.5) to capture reliable, large pseudo-labels that the main branch would otherwise ignore.

Self-Distillation (SD)

Because the high-resolution main branch and the low-resolution PIM branch capture complementary information, we employ a self-distillation module. By calculating the KL divergence between their classification predictions, we enable a mutually beneficial learning loop where both branches compensate for each other's weaknesses.

LSM is architecture-agnostic and can be integrated into both CNN-based (Faster R-CNN) and Transformer-based (Deformable-DETR) detectors.

Experimental Results

We evaluated LSM on MS-COCO, PASCAL VOC, and a new cross-domain COCO-ImageNet benchmark designed to test robustness against highly noisy unlabeled data.

  • MS-COCO: Under a 5% labeling ratio, Faster R-CNN with LSM achieved 31.97% mAP, a +3.54% improvement over the previous state-of-the-art (UBTeacher).
  • PASCAL VOC: LSM yielded a +4.21% mAP improvement over the baseline.
  • Cross-Domain Robustness: In the COCO-ImageNet setting, LSM applied to Deformable-DETR reached 45.34% mAP, outperforming the baseline by 1.3%. This demonstrates the method's ability to denoise out-of-distribution unlabeled data.
Qualitative examples of detections produced by the LSM method, showing accurate localization of objects even with moderate confidence scores.

Conclusion

LSM addresses a critical blind spot in current SSOD methods by recognizing that scale can serve as a proxy for pseudo-label quality. By using downsampling as a physical noise filter and employing self-distillation, the method successfully mines valuable low-confidence samples. While the dual-branch architecture increases computational overhead, the significant performance gains across various benchmarks and architectures demonstrate its effectiveness in advancing semi-supervised object detection.

Summary updated

Abstract

Reliable pseudo labels from unlabeled data play a key role in semi-supervised object detection (SSOD). However, the state-of-the-art SSOD methods all rely on pseudo labels with high confidence, which ignore valuable pseudo labels with lower confidence. Additionally, the insufficient excavation for unlabeled data results in an excessively low recall rate thus hurting the network training. In this paper, we propose a novel Low-confidence Samples Mining (LSM) method to utilize low confidence pseudo labels efficiently. Specifically, we develop an additional pseudo information mining (PIM) branch on account of low-resolution feature maps to extract reliable large area instances, the IoUs of which are higher than small area ones. Owing to the complementary predictions between PIM and the main branch, we further design self-distillation (SD) to compensate for both in a mutually learning manner. Meanwhile, the extensibility of the above approaches enables our LSM to apply to Faster-RCNN and Deformable-DETR respectively. On the MS-COCO benchmark, our method achieves 3.54% mAP improvement over state-of-the-art methods under 5% labeling ratios.
BibTeX
@inproceedings{liu2023low,
  author = {Guandu Liu and Fangyuan Zhang and Tianxiang Pan and Jun-Hai Yong and Bin Wang},
  title = {{Low-Confidence Samples Mining for Semi-supervised Object Detection}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2023},
  url = {https://openreview.net/forum?id=WiyVDnN7Lb},
}
Figure