A linear method for camera pair self-calibration and multi-view reconstruction with geometrically verified correspondences

Nikos Melanitis, Petros MaragosarXiv 2019 · OpenReview

Abstract

We examine 3D reconstruction of architectural scenes in unordered sets of uncalibrated images. We introduce a linear method to self-calibrate and find the metric reconstruction of a camera pair. We assume unknown and different focal lengths but otherwise known internal camera parameters and a known projective reconstruction of the camera pair. We recover two possible camera configurations in space and use the Cheirality condition, that all 3D scene points are in front of both cameras, to disambiguate the solution. We show in two Theorems, first that the two solutions are in mirror positions and then the relations between their viewing directions. Our new method performs on par (median rotation error $\Delta R = 3.49^{\circ}$) with the standard approach of Kruppa equations ($\Delta R = 3.77^{\circ}$) for self-calibration and 5-Point algorithm for calibrated metric reconstruction of a camera pair. We reject erroneous image correspondences by introducing a method to examine whether point correspondences appear in the same order along $x, y$ image axes in image pairs. We evaluate this method by its precision and recall and show that it improves the robustness of point matches in architectural and general scenes. Finally, we integrate all the introduced methods to a 3D reconstruction pipeline. We utilize the numerous camera pair metric recontructions using rotation-averaging algorithms and a novel method to average focal length estimates.

Introduction

Creating accurate 3D models from unordered image collections—a process known as Structure-from-Motion (SfM)—traditionally relies on knowing internal camera parameters, particularly focal length. In real-world scenarios, such as processing vacation photos or historical archives, this metadata is often missing or unreliable. The conventional "Gold Standard" pipeline involves a two-stage iterative process: solving Kruppa equations for focal lengths followed by the 5-Point algorithm for metric reconstruction. However, these methods are computationally intensive and highly sensitive to outlier correspondences. This paper presents a novel linear approach that unifies self-calibration and reconstruction while introducing robust geometric verification to handle noisy data.

Methodology

Unified Linear Self-Calibration

The core contribution is a linear method that simultaneously solves for focal lengths and the metric structure of a camera pair. By leveraging the Dual Image of the Absolute Conic (DIAC), the authors derive a system of linear equations that recovers both the focal lengths and the "plane at infinity." This approach bypasses the complexity of iterative solvers used in standard methods.

Summary of notation and P matrix representations used in the linear reconstruction framework.

Geometric Disambiguation and the Mirror Theorem

The mathematical formulation yields two potential solutions for the camera configuration. The authors prove two fundamental theorems demonstrating that these solutions are geometrically related as mirror positions. To resolve this ambiguity, the Cheirality condition is applied: physically valid 3D points must reside in front of both cameras. This logical check effectively identifies the correct spatial arrangement.

Robust Correspondence Verification via LIS

To mitigate the impact of erroneous point matches, the paper introduces a verification technique based on the Longest Increasing Subsequence (LIS) problem. In structured architectural scenes, point correspondences typically maintain their relative spatial ordering across images. By treating the consistency of $x$ and $y$ coordinates as an LIS problem, the method efficiently filters out outliers that violate these geometric constraints, providing a faster alternative to traditional RANSAC-based filtering.

Joint Confidence Count (Jcc) for Focal Length Averaging

When integrating multiple camera pair reconstructions, the authors employ a "Joint Confidence Count" (Jcc) rather than simple statistical medians. This method weights focal length estimates based on the degree of agreement across different camera pairs, significantly refining the global calibration.

Experimental Results

The proposed framework was evaluated using Strecha’s benchmark datasets (e.g., castle-P30, Fountain-P11) and custom datasets of architectural landmarks in Athens, such as the Parthenon.

  • Accuracy: The linear method achieved a median rotation error of 3.49°, outperforming the Kruppa + 5-Point baseline (3.77°).
  • Calibration Precision: The Jcc averaging method reduced focal length estimation error from 0.28 to 0.07, representing a 75% improvement in accuracy over standard median-based approaches.
  • Robustness: The LIS-based verification demonstrated near-perfect precision on architectural scenes, ensuring high-quality input for the reconstruction pipeline.

Conclusion

This work demonstrates that a linear approach to self-calibration is not only feasible but competitive with established iterative methods. By combining unified linear solvers with LIS-based geometric verification and robust averaging techniques, the authors provide a resilient pipeline for 3D reconstruction from uncalibrated image sets. Future research may explore extending this linear model to solve for additional internal parameters and improving global rotation registration convergence.

Summary updated

Abstract

We examine 3D reconstruction of architectural scenes in unordered sets of uncalibrated images. We introduce a linear method to self-calibrate and find the metric reconstruction of a camera pair. We assume unknown and different focal lengths but otherwise known internal camera parameters and a known projective reconstruction of the camera pair. We recover two possible camera configurations in space and use the Cheirality condition, that all 3D scene points are in front of both cameras, to disambiguate the solution. We show in two Theorems, first that the two solutions are in mirror positions and then the relations between their viewing directions. Our new method performs on par (median rotation error $\Delta R = 3.49^{\circ}$) with the standard approach of Kruppa equations ($\Delta R = 3.77^{\circ}$) for self-calibration and 5-Point algorithm for calibrated metric reconstruction of a camera pair. We reject erroneous image correspondences by introducing a method to examine whether point correspondences appear in the same order along $x, y$ image axes in image pairs. We evaluate this method by its precision and recall and show that it improves the robustness of point matches in architectural and general scenes. Finally, we integrate all the introduced methods to a 3D reconstruction pipeline. We utilize the numerous camera pair metric recontructions using rotation-averaging algorithms and a novel method to average focal length estimates.
BibTeX
@misc{melanitis2019linear,
  author = {Nikos Melanitis and Petros Maragos},
  title = {{A linear method for camera pair self-calibration and multi-view reconstruction with geometrically verified correspondences}},
  year = {2019},
  eprint = {1906.12075},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1906.12075},
}
Figure