SCMT: Self-Correction Mean Teacher for Semi-supervised Object Detection

Feng Xiong, Jiayi Tian, Zhihui Hao, Yulin He, Xiaofeng RenIJCAI 2022 · DOI

Abstract

Semi-supervised object detection (SSOD) relies on a Teacher-Student framework where pseudo-labels generated by a teacher model guide the training of a student model. However, these pseudo-labels are inherently noisy, often suffering from missing annotations, incorrect classifications, and inaccurate localization. Self-Correction Mean Teacher (SCMT) addresses this by dynamically re-weighting student proposals based on the teacher's confidence in both classification and localization. This soft-filtering approach prevents the student from learning from incorrect data, leading to state-of-the-art performance on the MS-COCO benchmark.

Introduction

In SSOD, the challenge lies in effectively utilizing a massive amount of unlabeled data. The standard approach uses a "Teacher" model to generate pseudo-labels for unlabeled images, which the "Student" model then learns to predict. A major bottleneck in this process is the quality of pseudo-labels. If the Teacher is uncertain, it may produce "garbage" data that misleads the Student.

Visualization of noisy pseudo-labels (a) and student proposals with confidence weights (b).

Existing methods typically use a "hard threshold" to discard low-confidence labels. However, this creates a trade-off: high thresholds ensure precision but sacrifice recall, while low thresholds introduce noise. SCMT moves beyond this binary choice by implementing a dynamic, soft-weighting system that evaluates the quality of every proposal.

Methodology

The SCMT framework introduces a Self-Correction Weight applied to the unsupervised loss of the student model. This weight is derived from a composite confidence score that accounts for both classification certainty and localization accuracy (measured by the Intersection over Union between the teacher’s refined box and the original proposal).

The SCMT framework architecture showing the dynamic re-weighting process across RPN and RCNN stages.

Multi-Stage Re-weighting

SCMT intervenes at two critical stages of the detection pipeline:

  1. RPN Stage: Weights the Region Proposal Network's classification loss. If the teacher identifies an anchor as likely background or poorly localized, the student’s loss for that anchor is down-weighted.
  2. RCNN Stage: Weights both classification and regression losses. This ensures the student does not attempt to "correct" its coordinates to match a teacher's box that is itself poorly localized.

The Teacher model is updated via an Exponential Moving Average (EMA) of the Student’s weights, creating a virtuous cycle where the Teacher becomes more accurate as the Student improves.

Experiments and Results

The researchers evaluated SCMT on the MS-COCO 2017 dataset across various data-scarcity scenarios. SCMT consistently outperformed previous state-of-the-art methods like Unbiased-Teacher and Softer-Teacher.

  • Low-Data Regime (1% COCO): SCMT achieved 23.09 mAP, a massive 14.04 point increase over the supervised baseline (9.05 mAP).
  • Standard Regime (10% COCO): It reached 35.42 mAP, outperforming the supervised baseline by over 11 points.
  • High-End Performance: Even with the full COCO dataset plus additional unlabeled images, SCMT added 4.93 mAP, reaching a final score of 42.56 mAP.

A notable finding was the significant boost in detecting small objects ($AR_s$), which are typically the most difficult to label accurately in noisy environments.

Conclusion

SCMT demonstrates that soft filtering via dynamic weighting is a superior alternative to hard thresholding in semi-supervised object detection. By addressing the misalignment between classification and localization confidence, SCMT allows models to extract more useful information from unlabeled data while remaining robust to noise. Future work may explore the application of these self-correction weights to one-stage and transformer-based detectors.

Summary updated

BibTeX
@inproceedings{xiong2022scmt,
  author = {Feng Xiong and Jiayi Tian and Zhihui Hao and Yulin He and Xiaofeng Ren},
  title = {{SCMT: Self-Correction Mean Teacher for Semi-supervised Object Detection}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2022},
  doi = {10.24963/ijcai.2022/207},
}
Figure