Transductive Weakly-Supervised Player Detection Using Soccer Broadcast Videos

Chris Andrew Gadde, C.v. JawaharWACV 2022

TL;DR

This paper tackles the challenging problem of player detection in soccer broadcast videos, which are rife with "domain noise" (e.g., referees, audience members) that confuses standard detectors. They introduce a novel transductive approach that leverages a small amount of manually labeled "reliable" player instances to adapt a pre-trained model, achieving significant performance gains. The authors also release a new, large-scale annotated dataset of FIFA World Cup broadcasts to facilitate future research.

What Problem Does This Paper Address?

Player detection is a crucial first step for many sports analytics tasks, like player recognition, tracking, and event analysis. While detecting players in controlled environments (like fixed camera feeds) is relatively well-studied, broadcast videos present a much harder challenge. These videos are easily accessible but suffer from significant "domain shift" and "domain noise." Domain shift means a model trained on one match might perform poorly on another due to variations in camera angles, lighting, and player appearances. Domain noise refers to the presence of non-player objects that look similar to players (e.g., coaches, referees, audience members wearing team colors) which can lead to many false positives, especially for the "player" class. Existing methods often struggle with this noise, and obtaining large amounts of labeled data for every new match is impractical.

Methods

The core of this paper is a transductive approach for player detection. This means they use unlabeled target data (the broadcast videos themselves) to improve a model's performance on that specific data. Here's a breakdown of their method:

  1. Inductive Phase: They start with a pre-trained YOLOv3 detector (a common object detection model) and use it to generate initial bounding box proposals for players in the broadcast videos. Crucially, they also extract visual features for each proposed bounding box using a separate person re-identification model.
  2. Identifying Reliable Predictions (The Novelty): This is where they tackle the domain noise. Instead of relying on simple confidence scores, they propose a multi-model greedy clustering approach to assign "domain labels" (player vs. non-player) to the bounding boxes.
  3. They build a similarity graph between bounding boxes based on their visual features.
  4. They train multiple clustering models (e.g., K-Means, Gaussian Mixture Models) on subsets of these features.
  5. A greedy clustering algorithm (inspired by LambdaCC) uses these models to group similar bounding boxes. The goal is to create clusters that are internally dense (all players) and externally sparse (don't include non-players).
  6. They then select "reliable" bounding boxes from these well-formed clusters, effectively filtering out the domain noise.
  7. Transductive Phase: The "reliable" bounding boxes identified in the previous step are used as a small, manually labeled dataset. They then fine-tune a copy of the YOLOv3 model (specifically, the detection and upsampling layers) using these reliable instances. This fine-tuning adapts the model to the specific domain of the broadcast video, leading to improved player detection.

Datasets

  • FIFA 2018 World Cup Broadcast Videos: The authors created and released a new, comprehensive dataset for this work. It consists of broadcast videos from three FIFA 2018 World Cup matches, featuring continuous, unedited footage. The dataset includes annotations for bounding boxes at a per-frame level, covering various camera views, player positions, and challenging scenarios like occlusion and truncation. This dataset is significantly larger and more representative of real-world broadcast conditions than previous ones.
  • MS COCO: The initial YOLOv3 detector was pre-trained on the MS COCO dataset, a standard large-scale object detection benchmark.
  • MARS: The person re-identification model was pre-trained on the MARS dataset.

Key Insights and Ideas

  • Domain Noise is the Primary Hurdle: The paper strongly emphasizes that the presence of non-player objects (referees, audience) is the main reason standard detectors fail on broadcast videos, not just general domain shift.
  • Instance-Level Domain Labels are Crucial: Unlike many domain adaptation methods that use image-level labels, this work highlights that for noisy data like broadcast videos, labeling individual instances (bounding boxes) as "player" or "not player" is essential for effective adaptation.
  • Clustering as a Noise Filter: The novel multi-model greedy clustering approach is a clever way to automatically identify reliable player instances from noisy initial detections, significantly reducing the manual annotation effort required.
  • Transductive Learning for Practical Adaptation: The transductive approach is well-suited for sports analysis, where new videos are constantly generated, and manual annotation for each is infeasible. Fine-tuning with a small set of reliable instances offers a practical solution.

Key Results

  • Significant Performance Improvement: The proposed transductive method achieved an average of 16 point improvement in mAP (mean Average Precision) compared to baselines and existing state-of-the-art methods on their new dataset.
  • Efficient Annotation: This improvement was achieved by annotating only around 100 samples per video as reliable instances.
  • Outperforms Supervised Methods: Their approach, even with limited domain-specific labels, outperformed supervised methods trained on larger, but less domain-specific, datasets.
  • Effective Noise Removal: Their clustering method demonstrated high True Positive Retention Ratio (TPR) and False Positive Removal Ratio (FPR), indicating it effectively filters out non-player objects while keeping true positives.
  • Qualitative Improvements: Visual examples show their method successfully avoids detecting non-players (like staff in team jerseys) that confuse other detectors.

Limitations and Future Work

  • Reliance on Initial Detector: The performance of the transductive model is still dependent on the initial proposals from the inductive YOLOv3 model. If the initial detector misses many players or generates very noisy bounding boxes, the clustering might struggle.
  • Annotation Effort: While significantly reduced, some manual annotation is still required to identify the "reliable" instances for the transductive phase.
  • Future Work: The authors suggest exploring further applications of their reliable detection system, such as generating heat-maps of player locations (which they demonstrate as an example application) and combining it with player recognition for more advanced analysis. They also mention the potential for using this system for action detection and tracking specific players.

Summary updated

BibTeX
@inproceedings{gadde2022transductive,
  author = {Chris Andrew Gadde and C.V. Jawahar},
  title = {{Transductive Weakly-Supervised Player Detection Using Soccer Broadcast Videos}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2022},
}
Figure