Automated Top View Registration of Broadcast Football Videos

Rahul Anand Sharma, Bharath Bhat, Vineet Gandhi, C. V. Jawahar

Abstract

In this paper, we propose a novel method to register football broadcast video frames on the static top view model of the playing surface. The proposed method is fully automatic in contrast to the current state of the art which requires manual initialization of point correspondences between the image and the static model. Automatic registration using existing approaches has been difficult due to the lack of sufficient point correspondences. We investigate an alternate approach exploiting the edge information from the line markings on the field. We formulate the registration problem as a nearest neighbour search over a synthetically generated dictionary of edge map and homography pairs. The synthetic dictionary generation allows us to exhaustively cover a wide variety of camera angles and positions and reduce this problem to a minimal per-frame edge map matching procedure. We show that the per-frame results can be improved in videos using an optimization framework for temporal camera stabilization. We demonstrate the efficacy of our approach by presenting extensive results on a dataset collected from matches of football World Cup 2014.

Introduction

In the realm of sports analytics, obtaining "top-view" or planimetric data is considered the gold standard. This perspective allows analysts to accurately track player positioning, team formations, and ball movement within a standardized coordinate system. However, current industry solutions, such as those provided by Prozone or Tracab, typically rely on expensive, multi-camera stadium installations that require significant manual labor for calibration and initialization.

Adapting these capabilities to standard broadcast video presents significant computer vision challenges. Broadcasts are highly dynamic, featuring frequent cuts between cameras and constant panning, tilting, and zooming. Furthermore, a football pitch is largely "textureless"—it consists primarily of green grass. Traditional registration methods that rely on distinct "keypoints" (like SIFT or SURF) often fail because there are insufficient unique points on the field, especially during tight shots. This paper proposes a fully automated system that overcomes these hurdles by focusing on global edge structures rather than local points.

Methodology

The researchers developed a robust four-stage pipeline to automate the registration process without manual intervention:

  1. Pre-processing: To isolate field markings, the system employs a Stroke Width Transform (SWT). This technique is specifically designed to detect the consistent width of painted lines, making it more robust to shadows and grass patterns than standard edge detectors. Additionally, a Faster-RCNN detector is used to mask out players and crowds, ensuring the registration is based solely on pitch geometry.
  2. Synthetic Dictionary Generation: To solve the data scarcity problem, the authors mathematically simulated over 100,000 variations in camera pan, tilt, and zoom. This created a massive dictionary of synthetic edge maps paired with their exact mathematical transformations (homographies).
  3. Nearest Neighbor Matching: For any incoming video frame, the system extracts its edge map and searches the synthetic dictionary for the closest match. The study found that HOG (Histogram of Oriented Gradients) features were particularly effective for this geometric matching task.
  4. Temporal Smoothing & Stabilization: To ensure a smooth visual output, the system uses a Markov Random Field (MRF) to maintain logical camera movement over time. An L1-norm optimization is then applied to remove high-frequency jitters, mimicking the steady movement of a professional camera operator.
The proposed registration pipeline: from preprocessing and feature computation to nearest neighbor search within a synthetic dictionary.

Experimental Results

The system was rigorously tested on a dataset comprising video sequences from 16 different matches of the 2014 FIFA World Cup. Performance was evaluated using Intersection over Union (IOU), measuring the overlap between the predicted pitch registration and the ground truth.

Key results include:

  • High Accuracy: The HOG-based approach achieved a mean IOU of 86% on real-world broadcast footage.
  • Feature Superiority: Interestingly, HOG features (85.8% accuracy) significantly outperformed features from a pre-trained AlexNet (66.1%). This suggests that standard deep learning models trained on natural objects do not inherently generalize to the abstract, geometric world of pitch markings.
  • Robustness: The system maintained high performance across challenging conditions, including heavy stadium shadows, motion blur, and varying lighting conditions across different venues.
Visual demonstration of the system's output: (left) player detection with bounding boxes and (right) the resulting registration onto a 2D top-view model.

Conclusion and Future Work

This research presents a significant step toward fully automated sports analytics from broadcast feeds. By leveraging synthetic data and edge-based matching, the proposed method eliminates the need for tedious manual initialization. While the current system is optimized for the standard wide "top-down" broadcast view, future iterations could extend this to tighter close-ups and other field-based sports like hockey or basketball. Integrating this registration with advanced player-tracking algorithms will eventually enable a fully autonomous, end-to-end analytics engine from a single video source.

Summary updated

Abstract

In this paper, we propose a novel method to register football broadcast video frames on the static top view model of the playing surface. The proposed method is fully automatic in contrast to the current state of the art which requires manual initialization of point correspondences between the image and the static model. Automatic registration using existing approaches has been difficult due to the lack of sufficient point correspondences. We investigate an alternate approach exploiting the edge information from the line markings on the field. We formulate the registration problem as a nearest neighbour search over a synthetically generated dictionary of edge map and homography pairs. The synthetic dictionary generation allows us to exhaustively cover a wide variety of camera angles and positions and reduce this problem to a minimal per-frame edge map matching procedure. We show that the per-frame results can be improved in videos using an optimization framework for temporal camera stabilization. We demonstrate the efficacy of our approach by presenting extensive results on a dataset collected from matches of football World Cup 2014.
BibTeX
@inproceedings{sharma2018automated,
  author = {Rahul Anand Sharma and Bharath Bhat and Vineet Gandhi and C. V. Jawahar},
  title = {{Automated Top View Registration of Broadcast Football Videos}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2018},
  url = {https://openreview.net/forum?id=GmQIknw-npw},
}
Figure