TL;DR
This paper introduces a novel "Temporal Feature Multiplexing Network" (TFMN) for real-time table tennis ball trajectory detection. By efficiently reusing features across time and incorporating Transformer modules for global context, the TFMN achieves high accuracy (96.8% detection, 89.1% localization) with a remarkably small model size (7.68 MB) and high frame rates (634.19 FPS), outperforming existing methods.
What Problem Does This Paper Address?
Analyzing table tennis trajectories is crucial for understanding player strategies and weaknesses. However, current methods often rely on subjective human observation, lacking objective data. Existing automated approaches struggle with the small size of table tennis balls, the need for real-time processing, and achieving a balance between model complexity and detection accuracy. This paper aims to bridge this gap by developing a lightweight yet highly accurate system for real-time table tennis ball tracking.
Methods
The core innovation is the "feature store & return" module. This module allows the network to store features from previous frames and reuse them as input for subsequent layers in the current frame. This temporal feature reuse significantly enhances efficiency and accuracy, especially for a lightweight architecture.
Key components include:
- Backbone Network: A ResNet-based convolutional neural network for initial feature extraction.
- Transformer Module: Integrated to process stored features, capture global context, and build long-range feature relationships, complementing the local features from the CNN. A "TokenLearner" mechanism is used to reduce the computational burden of the Transformer.
- Kalman Filter: Used in the "location information return module" to predict the ball's position in the current frame based on previous frames, improving localization accuracy.
- Temporal Feature Multiplexing Network (TFMN): The overall architecture that combines these modules to achieve efficient feature reuse across time.
Datasets
The authors created a dataset by extracting frames from 50 video clips of table tennis matches, each 5 minutes long, captured from two angles (horizontal and perpendicular to the table). This dataset was used for training and evaluating the proposed model and its variants.
Key Insights and Ideas
- Feature Reuse is Key for Lightweight Models: The paper demonstrates that by intelligently reusing features across temporal frames, a network can achieve high performance without needing excessive depth or parameters.
- Transformer for Global Context: Integrating Transformer modules, even in a lightweight manner, significantly enhances the network's ability to understand global relationships within the feature maps, crucial for accurate detection.
- Kalman Filter for Precise Localization: Combining the deep learning detection with a Kalman filter for temporal prediction provides a robust way to refine the ball's position, especially in continuous motion.
- Trade-offs in Network Design: The ablation studies highlight how different combinations of modules (feature storage, Transformer, Kalman filter) impact accuracy, speed, and complexity, guiding the selection of the optimal architecture.
Key Results
- High Detection Accuracy: The proposed TFMN achieved 96.8% detection accuracy for table tennis balls.
- Good Localization Accuracy: Target localization accuracy reached 89.1%.
- Extremely Lightweight Model: The model size is only 7.68 MB, making it suitable for deployment on resource-constrained devices.
- High Frame Rate: The network can process frames at a rate of 634.19 FPS, enabling real-time detection.
- Ablation Study Performance: Network 6, which incorporated all proposed modules (backbone, feature storage, lightweight Transformer, Kalman filter), showed the best overall performance, achieving 98.2% precision, 97.3% recall, 96.8% AP, and 89.1% IoU.
Limitations and Future Work
- Initial Frame Detection: The authors acknowledge that the detection accuracy at the very beginning of a frame sequence might not be ideal, as it relies on previous frame extraction.
- Future Work:
- Design a two-stage network where a high-level network performs frame-skipping detection to generate high-quality feature maps for a low-level network.
- Develop an analysis system to decompose and obtain opponent's attacking and defensive styles based on trajectory data.
- Provide real-time information for coaches to distinguish opponent's hitting styles and trajectories.
Summary updated