Automatic Camera Calibration Using Active Displays of a Virtual Pattern

Lei Tan, Yaonan Wang, Hongshan Yu, Jiang Zhu2017 · DOI

TL;DR

This paper introduces a fully automatic camera calibration method that replaces physical patterns with a virtual one displayed on a screen. This eliminates the time-consuming and error-prone manual step of moving a physical pattern, leading to more stable and accurate calibration results.

What Problem Does This Paper Address?

Camera calibration is a fundamental step in 3D computer vision, essential for recovering metric information from 2D images. The most common method, Zhang's method, relies on capturing images of a physical planar pattern (like a checkerboard). While parts of this process can be automated, a crucial manual step remains: moving either the camera or the pattern to capture multiple views. This manual operation is not only time-consuming but also introduces instability in the calibration results, as slight variations in motion or pattern positioning can lead to inaccuracies. To achieve stable calibration, many images (often 20 or more) are required to ensure uniform distribution of detected points across the image. Poor point distribution, as illustrated in Figure 1b, can significantly degrade calibration accuracy. This paper aims to overcome these limitations by proposing a fully automatic calibration method that removes the need for manual manipulation.

Methods

The core innovation of this paper is the replacement of a physical calibration pattern with a virtual one. This virtual pattern is actively transformed and displayed on a fixed screen. The camera, also fixed, captures images of this displayed pattern. The proposed method then estimates camera parameters by finding correspondences between the virtual 3D points of the pattern and their 2D projections on the image.

Key aspects of the method include:

  • Virtual Pattern Transformation: The virtual pattern is transformed in virtual world coordinates and projected onto a screen. This allows for precise control over the pattern's orientation and position, ensuring that the control points are uniformly distributed across the camera's view.
  • Fixed Camera and Screen: Unlike conventional methods, both the camera and the screen remain stationary throughout the calibration process. This eliminates motion blur and ensures consistent capture conditions.
  • Closed-Form Solution: Similar to Zhang's method, the paper utilizes a closed-form solution to estimate initial camera parameters. This involves deriving constraints from homography matrices derived from point correspondences. However, instead of using orthogonality constraints (as in Zhang's method), this approach uses ratio constraints of vector dot products.
  • Nonlinear Refinement: After the initial linear estimation, a nonlinear optimization step is performed to refine the intrinsic and extrinsic parameters, including lens distortion coefficients. This minimizes the re-projection error, leading to higher accuracy.

Datasets

The paper evaluates its method using both synthetic and real image data.

  • Synthetic Data: Simulated camera images are generated with known intrinsic parameters and varying levels of Gaussian noise. This allows for a controlled evaluation of the method's performance under different noise conditions and with varying numbers of calibration images.
  • Real Data: Experiments are conducted using a real camera (Microsoft Kinect for Windows V2 sensor) and a 24-inch LCD monitor to display the virtual pattern. The results are compared against calibration performed using a traditional physical checkerboard.

Key Insights and Ideas

  • Automation is Key: The most significant insight is that by using an actively controlled virtual pattern on a fixed screen, the entire calibration process can be fully automated, eliminating the manual steps that plague traditional methods.
  • Uniform Point Distribution: Actively transforming the virtual pattern allows for guaranteed uniform distribution of control points, which is crucial for robust and accurate calibration. This addresses a known weakness of conventional methods where manual movement can lead to clustered points.
  • Eliminating Motion Blur: Fixing both the camera and the screen prevents motion blur, leading to sharper images and more precise localization of control points. This is particularly important when dealing with real-world scenarios where camera movement can introduce significant errors.
  • Comparable Accuracy: The proposed method achieves accuracy comparable to the standard method by Zhang, while offering significant advantages in terms of speed and automation.

Key Results

  • Robustness to Noise: The method demonstrates good performance even with moderate levels of noise. For a noise standard deviation of $\sigma=0.5$ pixels, the relative error in focal length is less than 0.1%, and the absolute error in the principal point is around 1 pixel.
  • Impact of Image Number: The accuracy of the calibration improves significantly as the number of calibration images increases, stabilizing after approximately 3 images.
  • Real-World Performance: Experiments on real images show that the proposed method yields more consistent results with smaller standard deviations for camera parameters compared to the conventional method.
  • Lower RMSE: The proposed method achieved a lower mean RMSE (0.1855 pixels) compared to the conventional method (0.2337 pixels) in real-world experiments, indicating higher accuracy. The lowest RMSE achieved by the proposed method was 0.1460 pixels.

Limitations and Future Work

  • Screen Coverage: A primary limitation is the requirement that the entire camera view must be covered by the screen. For cameras with very large working distances or wide fields of view, a large screen might be necessary, which could be impractical.
  • Resolution Mismatch: In applications with very high-resolution cameras and short working distances, the screen's resolution might be lower than the camera's, leading to discretization issues and potential problems with corner point detection and localization. While high-resolution screens can mitigate this, it can still affect accuracy.
  • Future Work: The paper does not explicitly detail future work, but the identified limitations suggest potential avenues for research, such as developing methods for partial screen coverage or addressing resolution mismatches.

Summary updated

BibTeX
@misc{tan2017automatic,
  author = {Lei Tan and Yaonan Wang and Hongshan Yu and Jiang Zhu},
  title = {{Automatic Camera Calibration Using Active Displays of a Virtual Pattern}},
  year = {2017},
  doi = {10.3390/s17040685},
}

Loading related content...

Figure