TL;DR
This research introduces two novel minimal solvers for absolute camera pose estimation that require only one or two feature correspondences. By leveraging the scale and orientation data already provided by common feature detectors (like SIFT or ORB), these methods significantly accelerate robust estimation (RANSAC) and improve localization accuracy in challenging, high-outlier environments.
The Challenge of Absolute Pose Estimation
To determine a camera's 6DOF position and orientation in the world, computer vision systems typically match 2D features in an image to 3D points in a scene. The traditional "gold standard" for this task is the P3P (Perspective-Three-Point) algorithm, which requires at least three point correspondences.
However, absolute pose estimation is usually performed within a RANSAC loop to filter out incorrect matches (outliers). The computational cost of RANSAC grows exponentially with the number of points required per sample. In environments with high outlier ratios—common in difficult lighting or changing urban landscapes—P3P can become slow and unreliable. This paper identifies a missed opportunity: most modern feature detectors provide not just a 2D location, but also the scale and orientation of the feature. The authors explore how to use this "extra" information to solve the pose problem with fewer points, making the process faster and more robust.
Methodology
The authors derive novel constraints by linking absolute pose equations to the geometry of scaled and oriented features. They introduce two primary solvers:
- P2ORI (General Case): This solver determines the full 6DOF pose using only two oriented feature correspondences. It utilizes a Cayley rotation parameterization to transform the problem into a system of three quadratic equations, which can be solved with extreme efficiency.
- UP1SIFT (Gravity-Aware): This solver estimates pose using only one scaled and oriented feature, assuming the camera's "up" direction (gravity) is known. Since most modern smartphones and drones are equipped with Inertial Measurement Units (IMUs) that provide gravity data, this reduces the unknown degrees of freedom to four, allowing a single match to suffice.
The researchers also provide a principled method to decompose standard affine transformations into scale and orientation values, ensuring compatibility even when only raw affine frames are available.
Key Results and Performance
The methods were evaluated on standard, large-scale visual localization benchmarks, including Cambridge Landmarks and Aachen Day-Night v1.1.
Accuracy and Robustness
The proposed solvers demonstrate superior performance in high-outlier scenarios. In the notoriously difficult "Street" scene of the Cambridge Landmarks dataset, P2ORI achieved a median position error of 21 cm, compared to 197 cm for the traditional P3P. When integrated into a modern RANSAC framework (GC-RANSAC), the weighted average position error across all Cambridge scenes dropped from 120 cm (P3P) to just 14 cm (P2ORI) and 4 cm (UP1SIFT).
Efficiency
All proposed solvers are incredibly fast, executing in under 3 microseconds per trial. By reducing the minimal sample size from three points to one or two, the number of RANSAC iterations required to find a "clean" set of matches drops dramatically, particularly when 80-90% of matches are outliers.
Limitations and Sensitivity
While these solvers offer significant advantages in speed and robustness to outliers, they introduce new dependencies. Because they rely on the orientation and scale of a feature, they are more sensitive to noise in those specific measurements than traditional point-only solvers are to pixel noise.
As shown in the sensitivity analysis, if a detector provides a poor orientation estimate, the solver's accuracy degrades. Additionally, the solvers assume knowledge of the surface normal at the 3D point (often available from the point cloud). Accuracy can suffer if the scene geometry is highly complex or if the normals are poorly estimated.
Conclusion
By exploiting "free" metadata like scale and orientation, this work proves that physical point correspondences can be mathematically replaced to improve camera localization. The success in the "Street" scene suggests that orientation data provides critical geometric constraints that point locations alone lack. Future work may extend these solvers to uncalibrated cameras and real-time SLAM systems.
Summary updated