Physics-based ball tracking and 3D trajectory reconstruction with applications to shooting location estimation in basketball video

Hua-Tsung Chen, Min-Chun Tien, Yi-Wen Chen, Wen-Jiin Tsai, Suh-Yin LeeJVCIR 2009 · DOI

TL;DR

This paper presents a novel physics-based system for automatically tracking basketballs in 2D video and reconstructing their 3D trajectories. This allows for precise estimation of shooting locations, providing valuable data for game analysis and strategy development.

What Problem Does This Paper Address?

The increasing volume of sports video data necessitates automated analysis tools. Specifically, understanding game dynamics and player tendencies requires detailed statistics. While shot classification and highlight extraction are common, accurately tracking the ball and reconstructing its 3D path from a single 2D video stream, especially in fast-paced games like basketball, remains a significant challenge. This loss of depth information in 2D projection makes it difficult to determine precise shooting locations, a crucial statistic for coaches and analysts.

Methods

The authors propose an integrated system that leverages domain knowledge and the physical characteristics of ball motion to overcome the limitations of traditional 2D tracking. Key components include:

  • Court Shot Retrieval: The system first identifies "court shots" which provide a global view of the game, essential for tracking the entire trajectory. This involves shot boundary detection and classification into court, medium, and close-up views using color and spatial composition features.
  • Camera Calibration: To enable 3D reconstruction from 2D video, a robust camera calibration method is employed. This goes beyond standard 2D-to-2D transformations by incorporating non-coplanar feature points (like the backboard top-border) to capture height information and derive the 3D-to-2D geometric transformation.
  • Physics-Based Ball Tracking: This is the core innovation. Instead of relying solely on visual features, the system integrates physical constraints of ball motion (gravity, parabolic trajectory) into a dynamic programming approach. This helps in:
  • Ball Candidate Detection: Identifying potential ball pixels using color and motion, followed by shape and size sieves to filter out non-ball objects.
  • Trajectory Generation: Generating potential trajectories from ball candidates and then using a physics-based model to verify and select the most plausible shooting trajectories. This significantly reduces false positives compared to purely motion-based methods.
  • 3D Trajectory Reconstruction and Shooting Location Estimation: Once a 2D shooting trajectory is identified, the calibrated camera parameters and the physics-based trajectory model are used to reconstruct the 3D path. The starting point of this 3D trajectory is then projected onto the court model to estimate the shooting location.

Datasets

The system was evaluated on broadcast basketball video sequences from:

  • Olympics gold medal game: USA vs. Spain
  • Olympics game: USA vs. China
  • Taiwan high-school basketball league (HBL) game
  • Korea basketball game

The paper also mentions using 30 manually segmented basketball images to generate color histograms for ball detection. No new benchmark datasets were introduced.

Key Insights and Ideas

  • Physics as a Powerful Constraint: Integrating the known physics of ball motion (gravity, parabolic path) is highly effective in disambiguating true ball trajectories from false positives, especially in cluttered sports scenes.
  • 2D to 3D Reconstruction from Single View: The paper demonstrates that it's possible to achieve meaningful 3D reconstruction from a single camera view by carefully incorporating domain knowledge (court geometry) and physical principles.
  • Domain-Specific Feature Engineering: The use of court lines, backboard features, and the specific color properties of a basketball are crucial for robust detection and calibration.
  • Efficiency Gains from Physics: By discarding trajectories that violate physical laws early on, the system significantly reduces the number of potential trajectories to process, leading to higher efficiency.

Key Results

  • Ball Tracking Performance: The proposed physics-based method achieved high recall (average 92.83%) and precision (average 95.41%) in ball tracking across various datasets. It also generated fewer potential trajectories (#PT) compared to a Kalman filter-based method, indicating higher efficiency.
  • Shooting Location Estimation Accuracy: The system achieved an accuracy of 86.21% for shooting location estimation, judged by an experienced basketball player.
  • Qualitative Improvements: The paper highlights that incorporating vertical (height) information through 3D reconstruction leads to significantly more precise shooting location estimates compared to 2D-only projections.

Limitations and Future Work

  • Replay Shot Elimination: The paper mentions that replay shots are eliminated in advance by previous research, implying this is a prerequisite for their system.
  • Trajectory Splitting: A significant cause of tracking misses and shooting location estimation errors is when a shooting trajectory is split into two potential trajectories (e.g., when the ball goes out of frame and then reappears). The system retains only the trajectory approaching the backboard, which can lead to misses.
  • Occlusion and Calibration Errors: Misdetection of court lines or the backboard top-border due to occlusion (e.g., by player or captions) can lead to calibration errors and consequently, deviations in shooting location estimation.
  • Future Work: The authors are exploring the application of their physics-based motion models to other sports like volleyball and baseball. They believe their approach can lead to satisfactory solutions for automatic intelligence collection in various sports games.

Summary updated

BibTeX
@article{chen2009physics,
  author = {Hua-Tsung Chen and Min-Chun Tien and Yi-Wen Chen and Wen-Jiin Tsai and Suh-Yin Lee},
  title = {{Physics-based ball tracking and 3D trajectory reconstruction with applications to shooting location estimation in basketball video}},
  journal = {J. Vis. Commun. Image Represent.},
  year = {2009},
  doi = {10.1016/j.jvcir.2008.11.008},
}

Loading related content...

Figure