PTZ-Calib: Robust Pan-Tilt-Zoom Camera Calibration

Jinhui Guo, Lubin Fan, Bojian Wu, Jiaqi Gu, Shen Cao, Jieping Ye

Introduction

Pan-Tilt-Zoom (PTZ) cameras are essential components in modern surveillance, traffic monitoring, and sports broadcasting. Despite their ubiquity, achieving precise calibration—estimating the camera's pose, focal length, and lens distortion—remains a significant technical challenge. While PTZ hardware provides internal telemetry, these values are frequently inaccurate due to mechanical tolerances and environmental factors. PTZ-Calib introduces a novel two-stage framework designed to provide robust, high-precision calibration for any viewpoint in both planar and complex 3D environments.

Problem Statement

Existing PTZ calibration techniques generally fall into two categories, each with inherent limitations:

  1. Sports Field Registration: These methods typically assume the scene is a flat plane. While effective for football pitches, they fail in 3D environments like urban intersections where depth and geometry are critical.
  2. General PTZ Calibration: These approaches often rely on restrictive assumptions, such as the absence of lens distortion, a perfect rotation around a single point, or the requirement of a known initial position.

There is a clear need for a general-purpose calibration method that can handle lens distortion and complex 3D geometry while maintaining the speed required for real-world deployment.

Methodology

PTZ-Calib utilizes a two-stage pipeline that separates computationally intensive mapping from high-speed online calibration.

Offline Stage: Incremental Bundle Adjustment

The offline stage focuses on building a comprehensive map of the camera's environment:

  • Correspondence Search: The system captures a 360° set of reference images. It employs deep learning-based features, specifically SuperPoint and SuperGlue, to establish robust matching points across different views.
  • PTZ-IBA: The core contribution is the Incremental Bundle Adjustment (IBA) tailored for PTZ. Because PTZ cameras primarily rotate, traditional 3D triangulation is ineffective. Instead, the authors utilize "ray landmarks"—directional vectors originating from the camera center—to constrain relative rotation and intrinsic parameters.
  • Georeferencing: To align the local camera coordinates with real-world coordinates (e.g., GPS/UTM), the system incorporates a small number of manual 2D-3D annotations across a few reference images.

Online Stage: Real-Time Relocalization

Once the environment is mapped, calibration for any new viewpoint is treated as a relocalization problem. The current frame is matched against the pre-calibrated reference frames to estimate parameters. This "heavy lifting offline, light lifting online" architecture allows for near-instantaneous calibration during active operation.

Experimental Evaluation

Datasets

The framework was evaluated using two distinct sources:

  • WorldCup Dataset: A standard benchmark for sports registration consisting of 209 training and 186 test images from the 2014 World Cup.
  • Synthetic PTZ Dataset: To test 3D scene estimation, the authors generated 10 datasets using Blender, featuring indoor and outdoor scenes with ground-truth parameters and simulated Gaussian noise.

Key Results

  • Accuracy: On the WorldCup benchmark, PTZ-Calib achieved an Intersection over Union (IoU) of 96.8%, surpassing specialized state-of-the-art methods like SCCvSD and TVCalib.
  • 3D Robustness: In synthetic tests, PTZ-Calib significantly outperformed PTZ-SLAM. Notably, its focal length error was reduced to 2.13 pixels, compared to 20.25 pixels for PTZ-SLAM.
  • Efficiency: The online calibration stage averages 0.68 seconds per frame, making it suitable for real-time applications.

Discussion and Insights

A critical insight of this work is the use of ray landmarks over traditional 3D points. By modeling landmarks as rays, the system bypasses the translation requirements of standard Structure-from-Motion (SfM) algorithms, which are often violated by PTZ hardware. Furthermore, the explicit modeling of radial and tangential distortion allows the system to "straighten" wide-angle views that other models ignore.

Limitations and Future Work

Currently, the model assumes the camera rotates perfectly around its projection center. Future iterations aim to model the physical "lever arm" offset found in many mounts. Additionally, while the method reduces manual labor, the authors are exploring unsupervised georeferencing to eliminate the need for manual 2D-3D point matching entirely.

Summary updated

BibTeX
@inproceedings{guo2025ptz,
  author = {Jinhui Guo and Lubin Fan and Bojian Wu and Jiaqi Gu and Shen Cao and Jieping Ye},
  title = {{PTZ-Calib: Robust Pan-Tilt-Zoom Camera Calibration}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2025},
  doi = {10.1109/icra55743.2025.11128129},
}
Figure