TeamTrack: A Dataset for Multi-Sport Multi-Object Tracking in Full-pitch Videos

Atom Scott, Ikuma Uchida, Ning Ding, Rikuhei Umemoto, Rory P. Bunker, Ren Kobayashi, Takeshi Koyama, Masaki Onishi, Yoshinari Kameda, Keisuke FujiiWorkshops @ CVPR 2024

TL;DR

This paper introduces TeamTrack, a novel, large-scale dataset for multi-object tracking (MOT) in team sports, featuring full-pitch, multi-angle video from soccer, basketball, and handball. By addressing the limitations of existing datasets, TeamTrack aims to push the boundaries of MOT in complex, dynamic environments.

What Problem Does This Paper Address?

Multi-object tracking (MOT) is crucial for many applications, but it's particularly challenging in team sports. Existing MOT methods often struggle with objects that have similar appearances and exhibit diverse, complex movements, leading to frequent misdetections and ID switches. Furthermore, current datasets are often limited in scope, focusing on specific sports, using only partial views of the field (e.g., broadcast footage), or lacking sufficient diversity in scenarios. This gap hinders the development of robust MOT algorithms capable of accurately tracking players in the dynamic and often occluded environments of team sports.

Methods

The paper focuses on introducing and evaluating a new dataset. The core methodology involves:

  • Data Collection: Capturing high-resolution, full-pitch videos of soccer, basketball, and handball matches from multiple viewpoints (top-view via drones and side-view via fisheye lenses).
  • Annotation: Employing a semi-automatic annotation process using tools like CVAT and Labelbox, with manual adjustments and interpolation to ensure accuracy. This process required significant human effort (over 600 person-hours).
  • Dataset Characterization: Analyzing the dataset's characteristics, including appearance similarity and motion patterns, to highlight its unique challenges.
  • Benchmarking: Evaluating existing state-of-the-art MOT algorithms (ByteTrack and BoT-SORT) on the new dataset to establish baseline performance and demonstrate its utility. They also evaluated object detection (YOLOv8) and trajectory forecasting (Constant Velocity and LSTM models) as foundational tasks.

Datasets

The paper introduces the TeamTrack dataset, which is the primary contribution. It features:

  • Sports: Soccer, Basketball, Handball.
  • Views: Top-view (drone) and Side-view (fisheye lens).
  • Scale: Over 279,900 frames and 4.37 million bounding boxes, making it one of the largest datasets of its kind.

The paper also compares its results against existing MOT datasets such as:

  • MOT16, MOT20 (pedestrian-focused)
  • DanceTrack (dance-focused)
  • SoccerNet-Tracking, SportsMOT (sports-focused, but with limitations addressed by TeamTrack)

Key Insights and Ideas

  • Appearance Similarity is a Major Challenge: TeamTrack's analysis reveals a high degree of appearance similarity among players within the same team, making it difficult for algorithms to distinguish them, especially when combined with dynamic movements and occlusions.
  • Multi-View and Full-Pitch Coverage are Crucial: The dataset's full-pitch, multi-angle views enable research into multi-view tracking and the use of contextual information (like pitch dimensions) that are not possible with partial views.
  • Sports Dynamics Create Unique Tracking Challenges: The paper highlights that the inherent dynamics of team sports, with frequent occlusions, rapid motion, and inter-player interactions, pose significantly different challenges compared to pedestrian tracking datasets.
  • Annotation Effort is Substantial: The authors emphasize the considerable human effort required for annotating such a dataset, underscoring the need for more efficient annotation tools for future large-scale dataset creation.

Key Results

  • Object Detection: Fine-tuning YOLOv8 on TeamTrack significantly improved mAP scores across all sports and views. Top-view perspectives generally showed lower mAP, attributed to smaller player appearances from drone footage.
  • Trajectory Forecasting: LSTM models generally outperformed Constant Velocity models, validating the complex and non-linear motion patterns in TeamTrack.
  • Multi-Object Tracking: ByteTrack and BoT-SORT showed varying performance across different sports and views. Handball SideView yielded the highest HOTA scores, while Basketball SideView2 presented the most significant challenges. The dataset's variability across sports and views makes it a challenging benchmark.
  • Appearance Similarity: Quantitative analysis showed higher appearance similarity within TeamTrack compared to DanceTrack, MOT17, and MOT20.
  • Motion Patterns: TeamTrack exhibits more diverse motion patterns than pedestrian datasets, with basketball showing a higher frequency of relative position switches, indicating higher intensity.

Limitations and Future Work

  • Team and Venue Diversity: The authors acknowledge that the dataset has limitations regarding the diversity of teams and venues. Expanding the dataset to include a wider range of teams and locations could improve model generalization.
  • Annotation Tools: The significant annotation effort highlights the need for more efficient annotation tools to facilitate dataset expansion and scalability.
  • Future Research Directions: The paper suggests that incorporating team labels and models that capture player interactions (e.g., graph neural networks) could further improve tracking accuracy. The dataset is also expected to contribute to the development of more effective tracking models for complex sports scenarios.

Summary updated

BibTeX
@inproceedings{scott2024teamtrack,
  author = {Atom Scott and Ikuma Uchida and Ning Ding and Rikuhei Umemoto and Rory P. Bunker and Ren Kobayashi and Takeshi Koyama and Masaki Onishi and Yoshinari Kameda and Keisuke Fujii},
  title = {{TeamTrack: A Dataset for Multi-Sport Multi-Object Tracking in Full-pitch Videos}},
  booktitle = {Workshops @ CVPR},
  year = {2024},
}
Figure