Visual Image Recognition of Basketball Turning and Dribbling Based on Feature Extraction

Bowen Zhang, Tianqi WangTraitement 2022 · DOI

Abstract

The processing of basketball videos with complex contents faces several challenges in terms of global motion features, group motion features, and individual pose features. The current research cannot solve problems, such as the diverse spatiotemporal features of actions, the utilization of correspondence between spatiotemporal features, the increase of data volume, and the complexity of the network. To solve these problems, this paper studies the visual image recognition of basketball turning and dribbling based on feature extraction. Specifically, the optical flow image was introduced to establish the relationship between the velocity field of the basketball turning and dribbling and the grayscale of the image frame, such as to effectively depict the time variation of pixels. In addition, a convolutional neural network was established based on multi-feature learning to process the sports video image frames, and to extract more spatiotemporal features of basketball turning and dribbling. To improve the feature utilization of the action recognition model, this paper strengthens the extraction of dynamic and static features for the recognition of the player's basketball turning and dribbling in the same scene, and improves the existing convolutional neural network. Furthermore, the multi-feature learning of motion excitation and temporal aggregation of actions were completed. The proposed model was proved effective through experiments.

Introduction

Analyzing basketball video is a significant challenge for standard computer vision algorithms. Unlike sports with linear movements, basketball involves high-velocity changes, frequent mutual occlusion among players, and complex cooperative maneuvers. Standard models often struggle with the spatiotemporal aspect—the relationship between how a player looks (spatial) and how they move over time (temporal). Existing networks either become computationally overwhelmed by video data volume or fail to capture the subtle "micro-motions" of a crossover dribble or a pivot turn. This research aims to bridge this gap by creating a network that understands the "flow" of action through integrated feature extraction.

Methodology

The proposed system utilizes a dual-stream approach to focus on both static posture and dynamic motion.

Dual Backbone Architecture

To capture the velocity field of a player, the system uses optical flow, translating pixel movement between frames into grayscale values. This creates a visual map of motion. The architecture employs a dual backbone: ResNet101 for deep spatial feature extraction from RGB images, and a shallower BNInception network for processing optical flow images to reduce computational complexity.

Overall architecture of the proposed action recognition model showing the dual-stream processing of RGB and optical flow data.

Motion Excitation and Temporal Aggregation

The core innovation is a "squeeze-and-excitation" (SE) time aggregation module. Motion excitation calculates differences between adjacent frames at the feature level to highlight neural channels sensitive to movement. Temporal aggregation then divides these features into sub-series, processing them through a hierarchy of temporal and spatial sub-convolutions. This expands the model's receptive field in the time dimension, allowing it to analyze longer action sequences without losing detail.

Detailed structure of the squeeze-and-excitation and time aggregation module.

Cross-Modal Pre-training

Since optical flow data differs significantly from standard RGB images, cross-modal pre-training is used to discretize the optical flow field, enhancing the effectiveness of the initial convolutional layers in processing motion data.

Key Results

The study evaluated the model using professional basketball game videos. The optimized model achieved 91.02% accuracy on the primary dataset and up to 98.9% on a secondary test set. It significantly outperformed established models like ST-GCN (85.2%) and SGN (88.5%).

Confusion matrix analysis revealed high resilience to similar actions. The confusion rate between turning/dribbling and lay-up shots was only 1.45%. The most challenging distinction remained between reverse lay-ups and cross-leg dribbling (2.11% confusion rate) due to their similar physical profiles.

Conclusion

By combining optical flow analysis with a specialized temporal aggregation module, this research provides a robust framework for recognizing complex basketball actions. The dual-stream CNN effectively fuses spatial and temporal features, achieving high accuracy in a chaotic visual environment. Future work will focus on optimizing training times for the optical flow channel and testing the model's generalizability on lower-quality footage.

Summary updated

Abstract

The processing of basketball videos with complex contents faces several challenges in terms of global motion features, group motion features, and individual pose features. The current research cannot solve problems, such as the diverse spatiotemporal features of actions, the utilization of correspondence between spatiotemporal features, the increase of data volume, and the complexity of the network. To solve these problems, this paper studies the visual image recognition of basketball turning and dribbling based on feature extraction. Specifically, the optical flow image was introduced to establish the relationship between the velocity field of the basketball turning and dribbling and the grayscale of the image frame, such as to effectively depict the time variation of pixels. In addition, a convolutional neural network was established based on multi-feature learning to process the sports video image frames, and to extract more spatiotemporal features of basketball turning and dribbling. To improve the feature utilization of the action recognition model, this paper strengthens the extraction of dynamic and static features for the recognition of the player's basketball turning and dribbling in the same scene, and improves the existing convolutional neural network. Furthermore, the multi-feature learning of motion excitation and temporal aggregation of actions were completed. The proposed model was proved effective through experiments.
BibTeX
@inproceedings{zhang2022visual,
  author = {Bowen Zhang and Tianqi Wang},
  title = {{Visual Image Recognition of Basketball Turning and Dribbling Based on Feature Extraction}},
  booktitle = {Traitement},
  year = {2022},
  doi = {10.18280/ts.390624},
}
Figure