Automatic Calibration of a Urban Video Surveillance System through the Observation of Zebra Crossings

Alberto Broggi, A. Fascioli, Rean Isabella Fedriga, Stefano GhidoniResearch 2019 · DOI

TL;DR

This paper presents an automatic camera calibration method for urban surveillance systems that leverages the natural patterns of zebra crossings. By analyzing how these familiar road markings appear in stereo camera images, the system can accurately correct for lens distortion and camera orientation without requiring special calibration targets, making it robust for real-world, dynamic environments.

What Problem Does This Paper Address?

Accurate camera calibration is crucial for stereo vision systems, as errors can significantly degrade the performance of downstream algorithms. In urban surveillance, especially for applications like pedestrian monitoring at crosswalks, cameras are often mounted outdoors and are susceptible to shifts in position due to vibrations, temperature changes, and environmental factors. Traditional calibration methods can be time-consuming and require controlled environments or specific calibration patterns. This paper addresses the challenge of achieving robust and automatic camera calibration in such dynamic, real-world scenarios, where cameras might be slightly misaligned, and their lenses can introduce significant distortion (especially wide-angle or fish-eye lenses needed to capture a wide scene).

Methods

The proposed method consists of three main steps:

  1. Zebra Detector: This module identifies zebra crossing signs in the stereo images. It locates white rectangles, computes their edges, and selects those that are straight, long, and thin, and correspond to uniform, bright areas. Calibration points are then selected on the edges of these detected signs. The key insight here is using naturally occurring patterns rather than artificial ones.
  2. Distortion Removal: Using the selected calibration points, the system corrects for lens distortion. It employs the Field of View (FOV) model, which describes distortion as a radial movement dependent on the pixel's distance from the image center. A minimization algorithm is used to find the optimal distortion parameter that best rectifies the lines formed by the calibration points on the zebra crossings. This step generates a Look-Up Table (LUT) for efficient distortion correction.
  3. Perspective Removal: After distortion correction, Inverse Perspective Mapping (IPM) is applied to remove perspective effects. This technique requires camera intrinsic and extrinsic parameters. The paper proposes a novel method to estimate camera orientation by analyzing vanishing points formed by the intersection of zebra crossing sign sides. This allows the system to compensate for the camera's optical axis not being perfectly perpendicular to the ground, a common issue in outdoor setups. A further minimization step refines the camera orientation angles to ensure signs appear vertically aligned in the processed images.

Datasets

Not discussed in the paper. The system is designed to work with real-world images captured by the surveillance system itself, leveraging the presence of zebra crossings.

Key Insights and Ideas

  • Leveraging Natural Patterns: The core innovation is the use of ubiquitous zebra crossings as calibration targets, eliminating the need for artificial calibration patterns. This makes the system highly practical for real-world deployments.
  • Robustness to Real-World Conditions: The method is designed to handle significant lens distortion (e.g., from fish-eye lenses) and perspective effects arising from non-ideal camera mounting.
  • Automatic and Continuous Calibration: The automatic nature of the calibration process allows for frequent recalibration, which is essential for maintaining system accuracy in dynamic outdoor environments where camera positions can drift.
  • Combined Distortion and Perspective Correction: The sequential application of distortion removal followed by perspective removal provides a comprehensive approach to rectifying stereo images.

Key Results

The paper demonstrates that the system can achieve accurate calibration, even in challenging conditions.

  • Distortion Accuracy: The sum of squares of deviations for points after distortion correction is reported to be as low as 3.3.
  • Orientation Accuracy: The variance of angular coefficients for lines approximating calibration points is found to be negligible.
  • Image Alignment: The difference between the left and right stereo images after calibration is less than 3 pixels, indicating good alignment.
  • Performance in Difficult Cases: Even when a car obstructs some signs, reducing the number of calibration points, the system still provides acceptable results, demonstrating its robustness.

Limitations and Future Work

  • Limitations:
    • The system's performance is dependent on the presence and visibility of zebra crossings. If signs are heavily obscured or absent, calibration may fail.
  • The assumption of a flat road surface is critical for the Inverse Perspective Mapping (IPM) step; deviations from this assumption can lead to incorrect results.
  • The paper mentions that the perspective effect is "very weak" in their specific setup, which might imply that extreme perspective distortions could pose a greater challenge.
  • Future Work:
    • The paper suggests that a daily recalibration can solve problems arising from environmental changes, implying that continuous or scheduled recalibration is a natural extension.
  • While the method is robust, further investigation into handling more severe occlusions or variations in zebra crossing appearance could be beneficial.

Summary updated

Abstract

In this paper, a method for automatic calibration of a camera stereo pair through the observation of zebra crossing signs is described. It is based on the well-known consideration that it is possible to obtain information about lens distortion and camera orientation by observing how a known pattern appears in the image; moreover, a major advantage of this system is that it does not require any ad-hoc calibration pattern, because it exploits the zebra crossing signs, a pattern usually present in images used for monitoring pedestrians while crossing a road. To achieve this goal, well-known techniques for removing lens distortion and perspective effect are combined with new methods for locating calibration points on the available pattern, and, finally, for evaluating the camera position.
BibTeX
@inproceedings{broggi2019automatic,
  author = {Alberto Broggi and A. Fascioli and Rean Isabella Fedriga and Stefano Ghidoni},
  title = {{Automatic Calibration of a Urban Video Surveillance System through the Observation of Zebra Crossings}},
  booktitle = {Research},
  year = {2019},
  doi = {10.2390/biecoll-icvs2007-51},
}
Figure