Towards Structured Analysis of Broadcast Badminton Videos

Anurag Ghosh, Suriya Singh, C. V. Jawahar

TL;DR

This paper introduces an end-to-end framework for the automated analysis of broadcast badminton videos, transforming "unstructured" match footage into rich, actionable data. By combining deep learning-based player tracking with Temporal Convolutional Networks (TCNs) for stroke recognition, the authors can automatically calculate high-level metrics like player reaction time, speed, and court dominance without requiring specialized camera setups or sensors.

What Problem Does This Paper Address?

While sports analytics is a booming field, it remains bottlenecked by data acquisition. Currently, high-level analysis usually requires either expensive manual tagging by experts or specialized hardware (like Hawk-Eye systems or wearable sensors).

Broadcast video is a massive, untapped resource, but it is "unstructured." It contains replays, varying camera angles, and graphical overlays. Furthermore, badminton is a particularly challenging sport for computer vision: the action is incredibly fast-paced, players frequently occlude one another, and the "strokes" (actions) are intermittent and often involve subtle deceptions. The authors aim to bridge this gap by creating a system that can "understand" a standard match recording as well as a human coach might.

Methods

The authors propose a multi-stage pipeline to move from raw pixels to game statistics:

  1. Point Segmentation: To ignore replays and crowd shots, they extract HOG features and use a Support Vector Machine (SVM) combined with a Kalman filter to identify "play" segments where the camera is in the standard behind-the-baseline view.
  2. Player Tracking & Identification: They fine-tune a Faster R-CNN to detect the "Top" and "Bottom" players. To maintain player identity when they switch sides of the court, they use an unsupervised approach: extracting color histograms from player bounding boxes and clustering them using a Gaussian Mixture Model (GMM).
  3. Stroke Segmentation: This is the core of the action recognition. They utilize Temporal Convolutional Networks (TCNs)—specifically Encoder-Decoder TCNs and Dilated TCNs. Unlike standard RNNs, TCNs are better at capturing long-range dependencies and precise temporal boundaries in video sequences. They classify 12 categories (5 strokes per player + "react" and "none").
  4. Outcome Prediction: By analyzing the spatial positions of players in the first 50 frames of a point (leveraging badminton's service rules), they use a kernel SVM to predict which player won the point.

Datasets

The researchers introduced the Badminton Olympic Dataset, consisting of 27 matches from the 2012 London Olympics (retrieved from the official Olympic YouTube channel).

  • Scale: Over 15,000 annotated strokes and nearly 3,000 player bounding boxes.
  • Splits: 10 matches were manually annotated for training (7) and testing (3), ensuring no player overlap between sets to test the model's generalization to unseen athletes.

Key Insights and Ideas

  • The "React" Class: A clever addition to their stroke taxonomy is the "react" label. In racket sports, the time spent waiting and preparing for the opponent's return is as vital for analysis as the hit itself. This allows for the calculation of "Reaction Time" as a performance metric.
  • Rules as Features: Instead of relying solely on visual cues to see who won a point, the system exploits the "game grammar." Because players must stand in specific quadrants based on the score and who is serving, the model can infer the point outcome with high accuracy just by looking at player positioning at the start of a rally.
  • TCNs over RNNs: The paper reinforces the shift toward TCNs for temporal action localization, noting they are faster to train and more effective at handling the fast-paced, "bursty" nature of badminton strokes compared to traditional LSTMs.

Key Results

The framework achieved impressive benchmarks across all tasks:

  • Point Segmentation: 95.44% accuracy in distinguishing live play from replays.
  • Player Detection: 97.38% mAP, proving the robustness of the fine-tuned Faster R-CNN even with motion blur.
  • Player Identification: 97.98% accuracy in tracking players across court switches.
  • Stroke Segmentation: An edit score of 80.48%, which is quite high given the visual similarity between certain strokes (like a clear vs. a smash) in broadcast quality.
  • Outcome Prediction: 94.14% accuracy in determining the winner of a point based on initial positioning.

Limitations and Future Work

  • Visual Uniformity: The current player identification relies on color histograms. This fails in scenarios like Wimbledon, where all players wear white, or if both players happen to have similar jersey colors.
  • Deception: The model currently struggles with "deceptive" shots where a player intentionally masks their movement to fool an opponent. Detecting these would require even finer-grained pose estimation.
  • Camera Angles: The system currently discards non-baseline views (like side-court or overhead shots). Future work could involve multi-view fusion to improve tracking during heavy occlusion.
  • Forecasting: While the system is great at post-hoc analysis, the authors note that predicting a player's next move or position remains a significant "holy grail" challenge for the field.

Summary updated

Abstract

Sports video data is recorded for nearly every major tournament but remains archived and inaccessible to large scale data mining and analytics. It can only be viewed sequentially or manually tagged with higher-level labels which is time consuming and prone to errors. In this work, we propose an end-to-end framework for automatic attributes tagging and analysis of sport videos. We use commonly available broadcast videos of matches and, unlike previous approaches, does not rely on special camera setups or additional sensors. Our focus is on Badminton as the sport of interest. We propose a method to analyze a large corpus of badminton broadcast videos by segmenting the points played, tracking and recognizing the players in each point and annotating their respective badminton strokes. We evaluate the performance on 10 Olympic matches with 20 players and achieved 95.44% point segmentation accuracy, 97.38% player detection score ([email protected]), 97.98% player identification accuracy, and stroke segmentation edit scores of 80.48%. We further show that the automatically annotated videos alone could enable the gameplay analysis and inference by computing understandable metrics such as player's reaction time, speed, and footwork around the court, etc.
BibTeX
@inproceedings{ghosh2018towards,
  author = {Anurag Ghosh and Suriya Singh and C. V. Jawahar},
  title = {{Towards Structured Analysis of Broadcast Badminton Videos}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2018},
  url = {https://openreview.net/forum?id=kdZMGSwyWe_},
}

Loading related content...

Figure