A Two-point Method for PTZ Camera Calibration in Sports

Jianhui Chen, Fangrui Zhu, James J. Little

Abstract

Calibrating narrow field of view soccer cameras is challenging because there are very few field markings in the image. Unlike previous solutions, we propose a two-point method, which requires only two point correspondences given the prior knowledge of base location and orientation of a pan-tilt-zoom (PTZ) camera. We deploy this new calibration method to annotate pan-tilt-zoom data from soccer videos. The collected data are used as references for new images. We also propose a fast random forest method to predict pan-tilt angles without image-to-image feature matching, leading to an efficient calibration method for new images. We demonstrate our system on synthetic data and two real soccer datasets. Our two-point approach achieves superior performance over the state-of-the-art method.

Introduction

In the field of sports analytics and augmented reality broadcasting, accurately mapping 2D image pixels to 3D field coordinates is a fundamental requirement. This process, known as camera calibration or homography estimation, allows for the overlay of graphics (like offside lines) and the tracking of player movements in real-world units. Traditionally, this requires identifying at least four known point correspondences between the image and a field template.

However, professional sports broadcasts frequently utilize narrow field-of-view (FOV) shots to capture close-up action. In these scenarios, the camera is zoomed in so significantly that only a small portion of the pitch is visible, often containing fewer than the four required markings. Standard algorithms fail in these "feature-poor" environments. While database-retrieval methods exist, they are often slow and struggle with changes in lighting or stadium-specific advertisements. This paper introduces a "two-point method" that leverages the physical constraints of stadium-mounted cameras to overcome these limitations.

Methodology

The researchers' approach rests on two main innovations: a geometrically constrained algorithm and a regression-based prediction pipeline.

The Two-Point Algorithm

Most professional cameras in stadiums are bolted to fixed locations. By treating the camera's 3D position and base orientation as a known "static prior," the mathematical complexity of the calibration problem is reduced. The camera projection matrix is decomposed into this static component and a dynamic PTZ (Pan-Tilt-Zoom) component.

Visualization of the two-point method: (a) shows the field template with key points, and (b) shows the projected field lines on a real broadcast frame.

The authors prove that with this prior knowledge, the focal length can be solved using just two point correspondences, and the pan and tilt angles can subsequently be determined using only a single point. This reduction in required correspondences makes the system uniquely robust for zoomed-in shots where markings are sparse.

Pan-Tilt Forests and System Pipeline

To automate the calibration of new frames, the system moves away from traditional, slow image-to-image matching. Instead, it utilizes a "Pan-Tilt Forest"—a Random Forest regressor. This model is trained on local image features (SIFT descriptors) from a set of reference images. Given a patch from a new image, the forest predicts the corresponding camera "ray" (pan and tilt angles).

The system pipeline: Reference images are used to train a Pan-Tilt Forest, which then predicts angles for test images.

This shift from a search-based problem to a regression problem significantly boosts speed. Furthermore, because the underlying model only requires two points to fit, the RANSAC (Random Sample Consensus) loop used to filter out outliers (such as moving players) is much more efficient. While a standard 4-point method requires 71 iterations to reach a 99% success rate with 50% outliers, the 2-point method achieves the same reliability in just 16 iterations.

Experimental Evaluation

The proposed system was evaluated across three distinct datasets to test its robustness and accuracy: the Synthetic Stadium environment, a manually annotated "Highlights" dataset of 116 frames, and real-world footage from the 2014 World Cup.

Key Results

  • Accuracy: On the Highlights dataset, the method achieved a mean Intersection over Union (IoU) of 0.83, significantly outperforming the previous state-of-the-art method, CalibMe, which achieved 0.68.
  • Speed: The system is approximately 10 times faster than reference-based methods, processing frames in 0.3 seconds compared to 3.0 seconds.
  • Precision: Rotation errors were kept under 1 degree for over 85% of the tested images, maintaining high performance even in challenging narrow-FOV shots (under 25 degrees).

Conclusion and Future Work

The "two-point method" demonstrates that incorporating physical prior knowledge about camera placement can dramatically simplify the calibration problem in sports broadcasting. By reducing the degrees of freedom, the algorithm achieves state-of-the-art accuracy and real-time speeds, particularly in the difficult zoomed-in scenarios where traditional methods fail.

Future research directions include accounting for radial lens distortion, which can be significant in broadcast lenses, and exploring faster binary features (like LATCH) to replace SIFT. Additionally, while the current work focuses on soccer, the principles of the Pan-Tilt Forest could be extended to other field sports with distinct markings, such as basketball or American football.

Summary updated

Abstract

Calibrating narrow field of view soccer cameras is challenging because there are very few field markings in the image. Unlike previous solutions, we propose a two-point method, which requires only two point correspondences given the prior knowledge of base location and orientation of a pan-tilt-zoom (PTZ) camera. We deploy this new calibration method to annotate pan-tilt-zoom data from soccer videos. The collected data are used as references for new images. We also propose a fast random forest method to predict pan-tilt angles without image-to-image feature matching, leading to an efficient calibration method for new images. We demonstrate our system on synthetic data and two real soccer datasets. Our two-point approach achieves superior performance over the state-of-the-art method.
BibTeX
@inproceedings{chen2018two,
  author = {Jianhui Chen and Fangrui Zhu and James J. Little},
  title = {{A Two-point Method for PTZ Camera Calibration in Sports}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2018},
  url = {https://openreview.net/forum?id=miVdh6qLor},
}
Figure