ViSTec: Video Modeling for Sports Technique Recognition and Tactical Analysis

Yuchen He, Zeqing Yuan, Yihong Wu, Liqi Cheng, Dazhen Deng, Yingcai Wu

Introduction

Analyzing tactics in high-speed racket sports like table tennis, badminton, and tennis is traditionally a labor-intensive manual process. Professional teams rely on experts to log every stroke, a task complicated by motion blur, frequent occlusions, and the subtle physical differences between distinct techniques (e.g., a "topspin" vs. a "push"). Standard action recognition models often fail in this domain because they treat video frames as isolated events, ignoring the strategic dependencies—the "chess match"—where one stroke heavily influences the next.

Methodology: The ViSTec Framework

To address these challenges, we introduce ViSTec (Video-based Sports Technique recognition), a framework that mimics human cognition by combining visual perception with contextual logic. The architecture is divided into two primary modules:

1. Action Perception Module

This module handles the raw visual data. It utilizes a modified VideoMAE backbone optimized for high-speed sports. Unlike standard models that aggregate information over long temporal windows, ViSTec processes the video in 2-frame slices. This design choice is critical for capturing the "instantaneous" nature of a racket hitting a ball, preventing the micro-movements of a stroke from being "smeared" by temporal pooling. The pipeline operates in two stages: first, Stroke Segmentation identifies the exact timestamp of impact, and second, Stroke Classification determines the specific technique used.

2. Domain Knowledge Module ($grh$)

The core innovation of ViSTec is the integration of a directed graph representing domain-specific tactical knowledge. Nodes in this graph represent techniques, while edges represent the transition probabilities between them. By training the vision model and this graph jointly, the system gains a "contextual inductive bias." When visual evidence is ambiguous due to blur or occlusion, the model relies on the logical sequence of play to improve its prediction accuracy.

The ViSTec architecture overview, illustrating the two-stage pipeline from video input to technique classification using a domain knowledge graph.

Experimental Setup and Results

The framework was evaluated using the WTT Dataset, comprising 4,000 rally clips from 18 World Table Tennis games. Annotations were provided by professional athletes to ensure high-fidelity labeling of complex techniques.

ViSTec demonstrates significant performance gains over state-of-the-art action segmentation baselines such as MS-TCN, ASFormer, and UVAST. Key metrics include:

  • Frame-wise Accuracy: ViSTec achieved 83.5%, compared to 78.2% for the best-performing baseline (MS-TCN).
  • Edit Score: Measuring the correctness of the predicted sequence order, ViSTec reached 76.3, outperforming the nearest competitor (73.9).
  • Efficiency: The model achieves an inference speed of 39.3 FPS on a single A100 GPU, making it suitable for real-time broadcast analysis.

Tactical Analysis and Insights

Beyond simple recognition, ViSTec enables automated tactical discovery. By analyzing the predicted sequences, the system can quantify the effectiveness of specific strategies. For instance, data analysis confirmed professional intuition by identifying that the sequence "Serve → Short → Topspin" yields the highest scoring rate.

Tactical analysis enabled by ViSTec, showing scoring rates for specific stroke sequences and individual techniques.

Conclusion and Future Work

ViSTec successfully bridges the gap between low-level visual perception and high-level tactical reasoning in sports video analysis. While the current model excels at technique recognition, future iterations will aim to incorporate spatial features, such as ball placement and player positioning, to provide a more comprehensive tactical overview. Additionally, the researchers plan to explore personalized modeling to identify the "signature" patterns of world-class athletes.

Summary updated

Abstract

The immense popularity of racket sports has fueled substantial demand in tactical analysis with broadcast videos. However, existing manual methods require laborious annotation, and recent attempts leveraging video perception models are limited to low-level annotations like ball trajectories, overlooking tactics that necessitate an understanding of stroke techniques. State-of-the-art action segmentation models also struggle with technique recognition due to frequent occlusions and motion-induced blurring in racket sports videos. To address these challenges, We propose ViSTec, a Video-based Sports Technique recognition model inspired by human cognition that synergizes sparse visual data with rich contextual insights. Our approach integrates a graph to explicitly model strategic knowledge in stroke sequences and enhance technique recognition with contextual inductive bias. A two-stage action perception model is jointly trained to align with the contextual knowledge in the graph. Experiments demonstrate that our method outperforms existing models by a significant margin. Case studies with experts from the Chinese national table tennis team validate our model's capacity to automate analysis for technical actions and tactical strategies. More details are available at: https://ViSTec2024.github.io/.
BibTeX
@inproceedings{he2024vistec,
  author = {Yuchen He and Zeqing Yuan and Yihong Wu and Liqi Cheng and Dazhen Deng and Yingcai Wu},
  title = {{ViSTec: Video Modeling for Sports Technique Recognition and Tactical Analysis}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2024},
  doi = {10.1609/aaai.v38i8.28692},
}
Figure