Fast and Accurate Pose Estimation with Unknown Focal Length Using Line Correspondences

Kai Guo, Zhixiang Zhang, Zhongsen Zhang, Ye Tian, Honglin ChenSensors 2022 · DOI

TL;DR

This paper introduces a novel method for estimating a camera's 3D pose and unknown focal length using just two 2D-3D line correspondences and the camera's known position. By cleverly transforming the line-based problem into a point-based one, the method achieves high accuracy and speed, outperforming existing techniques in several benchmarks.

What Problem Does This Paper Address?

Estimating a camera's 3D pose (its position and orientation in space) is fundamental to many computer vision tasks like SLAM, photogrammetry, and augmented reality. Traditional methods often rely on matching 2D features in an image to their known 3D counterparts in the real world. While point correspondences are common, line correspondences offer a robust alternative, especially in scenarios where distinct points are scarce.

A significant challenge arises when the camera's focal length is unknown, which is common with zoom lenses. Estimating both pose and focal length simultaneously requires more data or more complex algorithms. Existing methods for line-based pose estimation (PnL) typically need at least three line correspondences to estimate pose with a known focal length, and even more if the focal length is unknown. This paper tackles the problem of achieving accurate and fast pose and focal length estimation using a minimal number of line correspondences (just two) by leveraging the prior knowledge of the camera's position.

Methods

The core innovation lies in a geometric transformation that converts the challenging perspective-n-line (PnL) problem into a more tractable perspective-n-point (PnP) problem. Here's a breakdown:

  • Focal Length Estimation:
    • The method utilizes the fact that a 3D line and the camera's position define a plane in 3D space. With two such lines, two planes are defined in both the world frame and the camera frame.
  • Crucially, the angle between these two planes is invariant under rigid body transformations (rotation and translation).
  • By calculating the angle between the normal vectors of these planes in both the world frame (where it's known) and the camera frame (where it depends on the unknown focal length), an equation is derived. This equation is a quadratic in $f^2$, allowing for efficient and direct estimation of the focal length.
  • Pose Estimation:
    • Once the focal length is known, the normal vectors of the planes in the camera frame can be precisely determined.
  • The paper then establishes a correspondence between the unit normal vectors of the planes in the world frame and the camera frame. This relationship is framed as a rotation matrix.
  • By translating the origin of the world frame to the camera's origin, the problem of transforming these normal vectors is converted into a standard 3D-3D point correspondence problem. This allows existing, efficient PnP solvers to be used for pose estimation.

This approach cleverly avoids iterative and nonlinear optimization, contributing to its speed and robustness.

Datasets

The paper uses synthetic data generated from a virtual perspective camera with a resolution of 1280x800. The synthetic data includes three thousand 2D-3D point and line correspondences. For real-world evaluation, images of a checkerboard with known dimensions were captured from two different perspectives using mobile phone cameras.

Key Insights and Ideas

  • Line-to-Point Transformation: The most significant insight is the elegant conversion of a PnL problem into a PnP problem by leveraging the geometric properties of planes formed by lines and the camera center. This is a powerful simplification.
  • Invariance of Plane Angles: The invariance of the angle between two planes under rigid transformations is a key geometric property exploited for focal length estimation.
  • Camera Position as a Constraint: Using the camera's known position as prior knowledge significantly reduces the number of required line correspondences, leading to faster and more broadly applicable solutions.
  • Direct Focal Length Estimation: Avoiding iterative methods for focal length estimation leads to better numerical stability and noise sensitivity.

Key Results

  • Accuracy: The proposed method demonstrates excellent performance in terms of numerical stability, noise sensitivity, and computational speed, often outperforming state-of-the-art methods like P3P, RPnP, GPnPf, DLT, and P3L.
  • Speed: It achieves the fastest computational speed among the compared methods, with a mean execution time of 0.46 ms per trial on a 3.3 GHz laptop. This is significantly faster than other methods, for example, 3.7 times faster than DLT.
  • Robustness: The method shows strong robustness to camera position noise, with minimal errors even when the camera position has up to 3 cm of noise.
  • Real-World Performance: Experiments with real images show competitive results, with the proposed method achieving the lowest mean relative position error (0.43%) among tested methods.

Limitations and Future Work

  • Limitations: The primary limitation acknowledged is the requirement for the camera's position to be known in advance. This might not always be feasible in all real-world scenarios.
  • Future Work:
    • Extend the method to estimate more intrinsic parameters (e.g., radial distortion) in addition to focal length and pose.
  • Integrate RANSAC with the proposed method to handle scenarios with a large number of line correspondences, including outliers, for applications like SfM and SLAM.
  • Investigate methods for extracting and optimizing imperfect lines, which are common in real-world images.

Summary updated

BibTeX
@article{guo2022fast,
  author = {Kai Guo and Zhixiang Zhang and Zhongsen Zhang and Ye Tian and Honglin Chen},
  title = {{Fast and Accurate Pose Estimation with Unknown Focal Length Using Line Correspondences}},
  journal = {Sensors},
  year = {2022},
  doi = {10.3390/s22218253},
}
Figure