Indoor Activity Detection and Recognition for Sport Games Analysis

Georg Waltner, Thomas Mauthner, Horst BischofarXiv 2014 · arXiv

Abstract

This paper presents a novel approach to recognizing individual player activities in volleyball by leveraging "Activity Context"—the collective behavior of all players on the court over time. By incorporating the actions of teammates and opponents, the proposed method improves classification accuracy for specific actions like "Reception" by over 18%. The study demonstrates that in structured team sports, individual actions are best understood through the lens of the group's spatial-temporal relationships.

Introduction

In computer vision, activity recognition has traditionally focused on isolated individuals. However, in structured environments like team sports, a person’s behavior is rarely independent. A volleyball player’s decision to "Block" is dictated by an opponent's "Attack," and a "Set" is usually preceded by a "Reception." While previous research has identified team-level activities, accurately identifying individual player actions within these complex, high-speed environments remains a challenge due to visual ambiguity and frequent occlusions. This research aims to bridge this gap by using the "rules of the game" and the spatial-temporal relationships between players as prior knowledge.

Methods

The authors propose a multi-stage pipeline for activity recognition:

Preprocessing and Localization

The system uses camera calibration (homography) to map image coordinates to real-world court coordinates. It employs Gaussian Mixture Models (GMMs) for color-based player segmentation and background subtraction to track player positions.

Player localization and court mapping using bounding boxes and ground plane segmentation.

Feature Extraction and Activity Context

For each player, the system extracts standard visual features including Histograms of Oriented Gradients (HOG) for shape and Histograms of Oriented Flow (HOF) for motion. Crucially, it incorporates Spatial Context (SC) descriptors that encode the player's exact location on the court and the overall distribution of all players.

Spatial distribution patterns for different volleyball activities, illustrating how location informs action classification.

The core innovation is the Activity Context (AC) Descriptor. After an initial Support Vector Machine (SVM) classifies individual frames, the AC descriptor aggregates these classification probabilities for all players over a sliding time window (typically 40 frames). This creates a "social" feature describing the state of the game surrounding a specific player. A final SVM then uses the combined visual features and the AC descriptor to produce the final activity label.

Experimental Results

The method was evaluated on a new dataset of 36,000 annotated frames from professional Austrian Volley League matches, covering seven categories: Stand, Service, Reception, Setting, Attack, Block, and Defense/Move.

Using only individual player features, the system achieved a baseline accuracy of 77.56%. The addition of the Activity Context descriptor provided a significant boost for volleyball-specific classes:

  • Reception: Improved by 18.35%, reaching 91.62% accuracy.
  • Block: Achieved 97.60% accuracy.
  • Service: Achieved 97.13% accuracy.

The "Activity Context" acts as a powerful temporal and social filter. For example, high "Attack" probabilities from players at the net significantly increase the likelihood that an opposing player is performing a "Block." However, general classes like "Defense/Move" (36.39% accuracy) remain challenging because they lack a specific signature and occur almost randomly.

Conclusion

This research demonstrates that incorporating group context is essential for accurate individual activity recognition in team sports. While the system handles some occlusion through context, future work will focus on incorporating more granular data, such as 3D skeletal information and the position of the ball, to further improve performance in "catch-all" activity categories and complex game scenarios.

Summary updated

Abstract

Activity recognition in sport is an attractive field for computer vision research. Game, player and team analysis are of great interest and research topics within this field emerge with the goal of automated analysis. The very specific underlying rules of sports can be used as prior knowledge for the recognition task and present a constrained environment for evaluation. This paper describes recognition of single player activities in sport with special emphasis on volleyball. Starting from a per-frame player-centered activity recognition, we incorporate geometry and contextual information via an activity context descriptor that collects information about all player's activities over a certain timespan relative to the investigated player. The benefit of this context information on single player activity recognition is evaluated on our new real-life dataset presenting a total amount of almost 36k annotated frames containing 7 activity classes within 6 videos of professional volleyball games. Our incorporation of the contextual information improves the average player-centered classification performance of 77.56% by up to 18.35% on specific classes, proving that spatio-temporal context is an important clue for activity recognition.
BibTeX
@misc{waltner2014indoor,
  author = {Georg Waltner and Thomas Mauthner and Horst Bischof},
  title = {{Indoor Activity Detection and Recognition for Sport Games Analysis}},
  year = {2014},
  eprint = {1404.6413},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1404.6413},
}
Figure