Tangent Sampson Error: Fast Approximate Two-view Reprojection Error for Central Camera Models

Mikhail Terekhov, Viktor LarssonICCV 2023 · DOI

1. Introduction

Estimating the relative pose between two cameras is a fundamental task in computer vision, typically solved by minimizing the reprojection error of point correspondences. The Maximum Likelihood (ML) reprojection error is considered the "gold standard" for accuracy but is computationally prohibitive for real-time applications like RANSAC due to the need for iterative non-linear optimization and 3D triangulation.

While the classical Sampson error offers a fast first-order approximation, it is strictly derived for the pinhole camera model. Modern vision systems frequently employ wide-angle, fisheye, or panoramic lenses that deviate significantly from pinhole geometry. This paper introduces the Tangent Sampson Error ($E_{TS}$), a generalized framework that extends the efficiency of Sampson-style approximations to any central camera model.

2. Methodology

The core of the Tangent Sampson Error lies in linearizing the projection manifold of arbitrary camera models. Unlike traditional methods that might require explicit image undistortion—a process that introduces resampling artifacts and is often impossible for ultra-wide fields of view (FOV)—$E_{TS}$ operates directly in the distorted image space.

2.1 Linearizing the Unprojection

The authors generalize the linearization process by considering the "unprojection" function, which maps 2D image coordinates to 3D bearing vectors. Since many complex camera models lack a closed-form inverse, the method utilizes the Inverse Function Theorem. The Jacobian of the unprojection is computed as the Moore-Penrose pseudoinverse of the forward projection Jacobian.

The Tangent Sampson error visualized as an epipolar band across different central camera models, including equirectangular (left) and fisheye (right) projections.

2.2 Computational Efficiency

A key advantage of this approach is that the Jacobians depend only on the initial 2D point locations. Consequently, they can be pre-computed once per correspondence. During iterative pose estimation (e.g., within a RANSAC loop), the error evaluation remains nearly as fast as the standard algebraic error while providing accuracy comparable to the full ML error.

3. Experimental Evaluation

The performance of $E_{TS}$ was validated using diverse datasets, including BabelCalib (ultra-wide FOV checkerboards) and real-world Structure-from-Motion (SfM) datasets like Grossmunster.

3.1 Accuracy and Consistency

The method demonstrates remarkable "pixel-space consistency." By operating in the original image coordinates, it allows for intuitive thresholding (e.g., setting a RANSAC inlier threshold in pixels) regardless of the specific lens distortion.

Comparison of error contours in a fisheye image (left) and its rectified counterpart (right). The Tangent Sampson Error maintains consistent geometric meaning across different projections.

3.2 Key Results

  • Speed: $E_{TS}$ is approximately 400x faster than the full ML reprojection error and only 2.2x slower than basic algebraic metrics.
  • Robustness: In RANSAC experiments with FOV $>150^\circ$, $E_{TS}$ consistently outperformed pinhole-based metrics, leading to more accurate rotation and translation estimates.
  • Convergence: When used for non-linear pose refinement, the Tangent Sampson Error provides a smoother cost surface, leading to better convergence properties than competing approximations.

4. Conclusion

The Tangent Sampson Error bridges the gap between fast algebraic errors and accurate ML reprojection errors for non-pinhole cameras. Its ability to handle arbitrary central camera models with minimal computational overhead makes it an ideal candidate for modern robust estimation pipelines. While currently limited to central models, its mathematical foundation provides a robust path forward for high-performance wide-angle vision systems.

Summary updated

BibTeX
@inproceedings{terekhov2023tangent,
  author = {Mikhail Terekhov and Viktor Larsson},
  title = {{Tangent Sampson Error: Fast Approximate Two-view Reprojection Error for Central Camera Models}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2023},
  doi = {10.1109/iccv51070.2023.00312},
}
Figure