Abstract
Broadcast sports field registration is traditionally addressed as a homography estimation task, mapping the visible image area to a planar field model, predominantly focusing on the main camera shot. Addressing the shortcomings of previous approaches, we propose a novel calibration pipeline enabling camera calibration using a 3D soccer field model and extending the process to assess the multiple-view nature of broadcast videos. Our approach begins with a keypoint generation pipeline derived from SoccerNet dataset annotations, leveraging the geometric properties of the court. Subsequently, we execute classical camera calibration through DLT algorithm in a minimalist fashion, without further refinement. Through extensive experimentation on real-world soccer broadcast datasets such as SoccerNet-Calibration, WorldCup 2014 and TS-WorldCup, our method demonstrates superior performance in both multiple- and single-view 3D camera calibration while maintaining competitive results in homography estimation compared to state-of-the-art techniques.
Introduction
In sports analytics, field registration—the process of mapping 2D broadcast video pixels onto a 3D field model—is fundamental. Accurate registration is essential for generating tactical heatmaps, offside lines, and player tracking data. However, broadcast video presents significant challenges: constant camera movement (pan, tilt, zoom), frequent occlusions by players, and feature-sparse views in close-up or oblique shots.
Traditional methods often rely on search-based approaches using large databases of pre-rendered views or standard homography estimation that ignores 3D camera geometry. These approaches frequently struggle with unique camera angles or depth perspective errors. To address these issues, we introduce PnLCalib, a pipeline that leverages geometric properties for robust sports field registration.
Methodology
PnLCalib treats field registration as a geometric optimization problem. The pipeline consists of several key stages, from data generation to a refined inference process.
Hierarchical Keypoint Generation
A core component of our method is the definition of a dense grid of keypoints. Beyond simple line intersections, we incorporate "extended" intersections (where lines would meet if continued), line-ellipse intersections (e.g., where the center circle meets the halfway line), and tangent points on field curves. This dense representation ensures that even in feature-sparse views, sufficient geometric constraints are available for calibration.
Inference and PnL Refinement
During inference, a modified HRNetv2 encoder-decoder network produces heatmaps for keypoints and field line extremities. An initial camera pose is computed using RANSAC and the Direct Linear Transformation (DLT) algorithm.
The system's primary innovation is the Point and Line (PnL) Refinement Module. Unlike systems that rely solely on point matching, PnLCalib performs non-linear optimization by minimizing a joint cost function. This function considers both the distance between detected points and their 3D counterparts, and the distance between detected line segments and projected 3D field lines. This dual approach "locks in" the calibration using the long, straight lines of the pitch, providing stability even when specific intersections are occluded.
Experimental Evaluation
We evaluated PnLCalib on three major soccer datasets:
- SoccerNet-Calibration (SN23): Over 22,000 images covering diverse broadcast perspectives.
- WorldCup 2014 (WC14): A standard benchmark for field registration.
- TS-WorldCup: A time-sequence dataset used to assess temporal stability.
Key Results
PnLCalib demonstrates state-of-the-art performance:
- Accuracy: On the SoccerNet test-center set, the refinement module improved the Jaccard Index (5-pixel threshold) from 74.4% to 80.6%.
- 3D Precision: On WC14, it achieved a projection error of 0.60 meters, outperforming previous methods (0.65m–0.84m).
- Efficiency: The system maintains respectable processing speeds (164ms to 439ms per frame), making it suitable for near-real-time applications.
Conclusion
By prioritizing geometric properties over database-heavy search methods, PnLCalib provides a robust solution for sports field registration. While it faces limitations with fisheye lens distortions and extreme close-ups lacking markings, its ability to leverage both points and lines sets a new standard for 3D camera calibration in broadcast sports. Future work will explore incorporating temporal consistency to handle moments of complete field obscurity.
Summary updated