Towards Structured Analysis of Broadcast Badminton Videos

Anurag Ghosh, Suriya Singh, C. V. Jawahar

Abstract

Sports video data is recorded for nearly every major tournament but remains archived and inaccessible to large scale data mining and analytics. It can only be viewed sequentially or manually tagged with higher-level labels which is time consuming and prone to errors. In this work, we propose an end-to-end framework for automatic attributes tagging and analysis of sport videos. We use commonly available broadcast videos of matches and, unlike previous approaches, does not rely on special camera setups or additional sensors. Our focus is on Badminton as the sport of interest. We propose a method to analyze a large corpus of badminton broadcast videos by segmenting the points played, tracking and recognizing the players in each point and annotating their respective badminton strokes. We evaluate the performance on 10 Olympic matches with 20 players and achieved 95.44% point segmentation accuracy, 97.38% player detection score ([email protected]), 97.98% player identification accuracy, and stroke segmentation edit scores of 80.48%. We further show that the automatically annotated videos alone could enable the gameplay analysis and inference by computing understandable metrics such as player's reaction time, speed, and footwork around the court, etc.

Introduction

While sports analytics is a booming field, it remains bottlenecked by data acquisition. High-level analysis typically requires either expensive manual tagging by experts or specialized hardware like Hawk-Eye systems or wearable sensors. Broadcast video represents a massive, untapped resource, but it is inherently "unstructured," containing replays, varying camera angles, and graphical overlays.

Badminton presents unique challenges for computer vision: the action is extremely fast-paced, players frequently occlude one another, and "strokes" are intermittent and often involve subtle deceptions. This paper introduces an end-to-end framework that transforms raw broadcast footage into actionable data, enabling the calculation of high-level metrics such as player reaction time, speed, and court dominance.

Automated annotation of a live match, showing player tracking, dominance metrics, and stroke timelines.

Methodology

The proposed framework utilizes a multi-stage pipeline to extract structured information from raw pixels:

  1. Point Segmentation: To filter out non-play segments (replays, crowd shots), the system extracts HOG features and employs a Support Vector Machine (SVM) combined with a Kalman filter to identify standard behind-the-baseline "play" views.
  2. Player Tracking & Identification: A Faster R-CNN is fine-tuned to detect "Top" and "Bottom" players. To maintain identity during court switches, an unsupervised approach uses color histograms and Gaussian Mixture Model (GMM) clustering.
  3. Stroke Segmentation: This core component uses Temporal Convolutional Networks (TCNs)—specifically Encoder-Decoder and Dilated TCNs—to classify 12 categories, including five strokes per player, a "react" phase, and a "none" class. TCNs are preferred over RNNs for their ability to capture long-range dependencies and precise temporal boundaries.
  4. Outcome Prediction: Leveraging badminton's service rules, a kernel SVM analyzes player positions in the first 50 frames of a point to predict the winner.
Overview of the end-to-end framework pipeline from raw video to match statistics.

Dataset: The Badminton Olympic Dataset

The researchers introduced a new benchmark consisting of 27 matches from the 2012 London Olympics. The dataset includes over 15,000 annotated strokes and nearly 3,000 player bounding boxes. For evaluation, 10 matches were manually annotated, with a strict split to ensure no player overlap between training and testing sets, testing the model's generalization to unseen athletes.

Key Insights and Metrics

A significant contribution of this work is the introduction of the "React" class. In racket sports, the time spent preparing for an opponent's return is as critical as the hit itself. By explicitly labeling this phase, the system can calculate "Reaction Time" as a performance metric.

Furthermore, the system exploits "game grammar." Because player positioning is dictated by the score and service rules, the model can infer point outcomes with high accuracy (94.14%) simply by observing initial positioning, rather than relying solely on complex visual cues of the final shot.

Results

The framework demonstrates robust performance across all automated tasks:

  • Point Segmentation: 95.44% accuracy.
  • Player Detection & Identification: 97.38% mAP for detection and 97.98% accuracy for identification across court switches.
  • Stroke Segmentation: Achieved an edit score of 80.48%, successfully distinguishing visually similar actions like clears and smashes in broadcast quality.

Conclusion and Future Work

While successful, the framework has limitations. Player identification currently relies on color, which may fail if jersey colors are identical. Additionally, detecting "deceptive" shots remains a challenge requiring finer-grained pose estimation. Future work will explore multi-view fusion to handle occlusions and the "holy grail" of sports analytics: forecasting a player's next move or position.

Summary updated

Abstract

Sports video data is recorded for nearly every major tournament but remains archived and inaccessible to large scale data mining and analytics. It can only be viewed sequentially or manually tagged with higher-level labels which is time consuming and prone to errors. In this work, we propose an end-to-end framework for automatic attributes tagging and analysis of sport videos. We use commonly available broadcast videos of matches and, unlike previous approaches, does not rely on special camera setups or additional sensors. Our focus is on Badminton as the sport of interest. We propose a method to analyze a large corpus of badminton broadcast videos by segmenting the points played, tracking and recognizing the players in each point and annotating their respective badminton strokes. We evaluate the performance on 10 Olympic matches with 20 players and achieved 95.44% point segmentation accuracy, 97.38% player detection score ([email protected]), 97.98% player identification accuracy, and stroke segmentation edit scores of 80.48%. We further show that the automatically annotated videos alone could enable the gameplay analysis and inference by computing understandable metrics such as player's reaction time, speed, and footwork around the court, etc.
BibTeX
@inproceedings{ghosh2018towards,
  author = {Anurag Ghosh and Suriya Singh and C. V. Jawahar},
  title = {{Towards Structured Analysis of Broadcast Badminton Videos}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2018},
  url = {https://openreview.net/forum?id=kdZMGSwyWe_},
}
Figure