Semi-Supervised Training to Improve Player and Ball Detection in Soccer

Renaud Vandeghen, Anthony Cioppa, Marc Van Droogenbroeck

Introduction

Deep learning has fundamentally changed sports analytics, yet its effectiveness remains tethered to the availability of large, high-quality annotated datasets. In soccer, accurately detecting players, referees, and the ball is crucial for automated tactical analysis and broadcasting. However, manual annotation is a significant bottleneck—it is labor-intensive, expensive, and often fails to bridge the "domain gap" between generic training data and the specific visual conditions of various stadiums and broadcasts.

Researchers from the University of Liège have proposed a semi-supervised learning framework designed to overcome this data scarcity. By leveraging massive amounts of unlabeled broadcast video through a novel "teacher-student" architecture, they have established a new state-of-the-art benchmark for object detection in soccer.

The Teacher-Student Framework

The core of the proposed method is a semi-supervised pipeline that allows a model to learn from its own predictions on unlabeled data. The process begins with a Teacher model—a standard Faster R-CNN detector—trained on a limited set of manually labeled images. This Teacher then performs inference on a large corpus of unlabeled soccer footage, generating "pseudo-labels" (predicted bounding boxes and classes) for every frame.

Overview of the Teacher-Student semi-supervised learning pipeline.

A second model, the Student, is then trained using a combination of the original ground-truth labels and these generated pseudo-labels. To ensure the Student does not simply memorize the Teacher's errors, the framework incorporates an iterative update cycle where the Student can eventually surpass the Teacher and take over its role for further refinement.

Managing Uncertainty: The "Doubt" Mechanism

The primary challenge in semi-supervised learning is the noise inherent in pseudo-labels. If a Student is forced to learn from incorrect Teacher predictions (false positives), its performance will degrade. To mitigate this, the authors introduced three distinct loss parametrizations to handle prediction uncertainty:

  • Single Threshold: Only pseudo-labels exceeding a high confidence threshold ($\tau_h$) are utilized.
  • Double Threshold and Doubt: This introduces a "doubt zone" between a low threshold ($\tau_l$) and a high threshold ($\tau_h$). Predictions falling within this range are ignored during training—they are treated as neither objects nor background.
  • Progressive Doubt: This strategy uses a linear ramp for the loss weight. As the Teacher's confidence increases from $\tau_l$ to $\tau_h$, the Student "trusts" the label more, gradually increasing the weight of the corresponding loss.
Visualizing the three loss parametrization strategies: Single Threshold, Double Threshold (Doubt), and Progressive Doubt.

This "doubt" mechanism is the critical innovation of the paper, preventing the Student from being misled by ambiguous detections while still allowing it to benefit from the vast amount of unlabeled data.

Datasets and Benchmarking

The study utilized the SoccerNet-v3 dataset, which contains over 340,000 human-annotated bounding boxes across nearly 25,000 frames. For the semi-supervised component, the authors extracted approximately 1.6 million frames from 435 hours of unlabeled broadcast footage. The models were evaluated on six key classes: player, goalkeeper, main referee, side referee, staff, and ball.

Key Results and Insights

The semi-supervised approach yielded significant performance gains, particularly in data-constrained scenarios. When using only 1% of the labeled data, the method improved the mean Average Precision (mAP) from 18.1% to 26.2%. With the full dataset, the "progressive doubt" strategy helped establish a new benchmark on SoccerNet-v3 with an mAP of 52.3%, a 1.3% improvement over the purely supervised baseline.

A crucial finding was the importance of a final fine-tuning phase. By training the Student exclusively on high-quality labeled data at the end of the process, the researchers were able to "clean up" noise learned from the pseudo-labels, consistently resulting in a performance jump regardless of the initial training strategy.

Conclusion

This research demonstrates that semi-supervised learning, combined with a robust mechanism for handling uncertainty, can significantly enhance object detection in sports. While the current method effectively addresses false positives through the "doubt" zone, future work aims to incorporate temporal information from video to tackle "false negatives"—objects the Teacher misses entirely. By looking at adjacent frames, future models could "fill in the blanks," leading to even more resilient and accurate soccer analytics tools.

Summary updated

Abstract

Accurate player and ball detection has become increasingly important in recent years for sport analytics. As most state-of-the-art methods rely on training deep learning networks in a supervised fashion, they require huge amounts of annotated data, which are rarely available. In this paper, we present a novel generic semi-supervised method to train a network based on a labeled image dataset by leveraging a large unlabeled dataset of soccer broadcast videos. More precisely, we design a teacher-student approach in which the teacher produces surrogate annotations on the unlabeled data to be used later for training a student which has the same architecture as the teacher. Furthermore, we introduce three training loss parametrizations that allow the student to doubt the predictions of the teacher during training depending on the proposal confidence score. We show that including unlabeled data in the training process allows to substantially improve the performances of the detection network trained only on the labeled data. Finally, we provide a thorough performance study including different proportions of labeled and unlabeled data, and establish the first benchmark on the new SoccerNet-v3 detection task, with an mAP of 52.3%. Our code is available at https://github.com/rvandeghen/SST .
BibTeX
@inproceedings{vandeghen2022semi,
  author = {Renaud Vandeghen and Anthony Cioppa and Marc Van Droogenbroeck},
  title = {{Semi-Supervised Training to Improve Player and Ball Detection in Soccer}},
  booktitle = {Workshops @ CVPR},
  year = {2022},
  doi = {10.1109/cvprw56347.2022.00392},
}
Figure