OSL-ActionSpotting: A Unified Library for Action Spotting in Sports Videos

Yassine Benzakour, Bruno Cabado, Silvio Giancola, Anthony Cioppa, Bernard Ghanem, Marc Van Droogenbroeck

Introduction

In the rapidly evolving field of sports analytics, action spotting has emerged as a critical task. Unlike temporal action localization, which identifies the duration of an activity, action spotting requires pinpointing the exact millisecond a pivotal event occurs—such as a goal, a foul, or a serve. Despite its importance, the research landscape has historically been fragmented. Researchers often develop bespoke, isolated codebases for specific algorithms, leading to "codebase fatigue." This lack of standardization makes it difficult to compare methods fairly, integrate new techniques into production pipelines, or scale analysis across different sports.

OSL-ActionSpotting is a unified Python library designed to consolidate these fragmented algorithms into a single, modular framework. By providing a standardized environment, it allows the research community to move beyond one-off implementations toward a more collaborative and reproducible ecosystem.

A Modular Architecture for Action Spotting

To address the diversity of existing methods, OSL-ActionSpotting adopts a modular architecture that decomposes the action spotting pipeline into three functional components:

  1. Backbone: This layer handles feature extraction, transforming raw video frames into high-dimensional mathematical representations. It typically utilizes Convolutional Neural Networks (CNNs) like ResNet.
  2. Neck: This intermediate component refines extracted features, with a specific focus on temporal context—analyzing how frames relate to one another over time to better understand the flow of the game.
  3. Head: The final layer is responsible for classification and localization, identifying the specific action type and assigning the precise timestamp.

This "plug-and-play" design allows researchers to swap individual components—for instance, replacing a CNN backbone with a Transformer-based one—without necessitating a complete rewrite of the spotting logic.

Integrated Methodologies

The library currently supports three cornerstone methods in the action spotting literature:

  • Temporally-Aware Learnable Pooling: Focuses on the spatial and temporal analysis of pre-extracted features to identify key moments.
  • Context-Aware Loss Function (CALF): Employs a segmentation-based approach to model the "before and after" states of an action, improving the model's understanding of event boundaries.
  • Precise Temporal Spotting (PTS): An end-to-end trainable method that learns spatio-temporal features directly from raw video data, rather than relying solely on pre-computed features.

Overcoming the Data Bottleneck

A significant technical hurdle in sports video processing is the I/O bottleneck. Traditionally, video decoding occurs on the CPU, which often cannot keep pace with the processing power of modern GPUs. OSL-ActionSpotting integrates NVIDIA DALI (Data Loading Library) to move video decoding, cropping, and resizing directly onto the GPU.

Furthermore, the library introduces a unified JSON format for datasets. Inspired by the COCO format used in object detection, this standard simplifies the process of porting new sports (such as basketball or tennis) into the library. Currently, the framework is validated against SoccerNet-v2, the industry standard featuring over 110,000 annotated actions across 500 professional football matches.

Key Results and Efficiency Gains

The primary goal of OSL-ActionSpotting was to prove that a unified framework could replicate state-of-the-art results while improving operational efficiency.

  • Reproducibility: The library successfully matched the "Loose" and "Tight" Average Mean Precision (mAP) of the original disparate codebases. For example, the PTS method achieved a 72.2% Loose mAP and 66.2% Tight mAP on SoccerNet-v2 within the unified environment.
  • Computational Speed: The integration of NVIDIA DALI resulted in a 33% reduction in training time and a 50% speedup in validation steps compared to traditional OpenCV-based loaders.
  • Storage Efficiency: By decoding video on-the-fly, the library eliminates the need to store pre-extracted frames, which can consume up to 10x more storage space than the original video files.

Conclusion and Future Work

OSL-ActionSpotting provides a much-needed "North Star" for the sports analytics community, offering a reliable, high-performance foundation for future research. Built on PyTorch Lightning, it is designed for accessibility and scalability.

While the current version is optimized for football, future iterations aim to demonstrate the library's versatility across a broader range of sports. The authors invite the community to contribute new modules, particularly Transformer-based architectures and multi-modal approaches that incorporate audio data, to further push the boundaries of what is possible in automated sports understanding.

Summary updated

Abstract

Action spotting is crucial in sports analytics as it enables the precise identification and categorization of pivotal moments in sports matches, providing insights that are essential for performance analysis and tactical decision-making. The fragmentation of existing methodologies, however, impedes the progression of sports analytics, necessitating a unified codebase to support the development and deployment of action spotting for video analysis. In this work, we introduce OSL-ActionSpotting, a Python library that unifies different action spotting algorithms to streamline research and applications in sports video analytics. OSL-ActionSpotting encapsulates various state-of-the-art techniques into a singular, user-friendly framework, offering standardized processes for action spotting and analysis across multiple datasets. We successfully integrated three cornerstone action spotting methods into OSL-ActionSpotting, achieving performance metrics that match those of the original, disparate codebases. This unification within a single library preserves the effectiveness of each method and enhances usability and accessibility for researchers and practitioners in sports analytics. By bridging the gaps between various action spotting techniques, OSL-ActionSpotting significantly contributes to the field of sports video analysis, fostering enhanced analytical capabilities and collaborative research opportunities. The scalable and modularized design of the library ensures its long-term relevance and adaptability to future technological advancements in the domain.
BibTeX
@misc{benzakour2024osl,
  author = {Yassine Benzakour and Bruno Cabado and Silvio Giancola and Anthony Cioppa and Bernard Ghanem and Marc Van Droogenbroeck},
  title = {{OSL-ActionSpotting: A Unified Library for Action Spotting in Sports Videos}},
  year = {2024},
  eprint = {2407.01265},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2407.01265},
}
Figure