Abstract: Estimating the global pose of multiple cameras and planar objects is a fundamental challenge in computer vision, particularly when dealing with large-scale environments where data is sparse. This paper presents a robust framework that leverages factorization and a novel "chaining" mechanism to solve for global consistency. By treating missing rotations as a graph problem, the algorithm successfully reconstructs 3D scenes even when over 80% of the camera-plane observations are missing. Experimental validation on both controlled and real-world indoor datasets demonstrates high angular precision and significant robustness to feature noise.
Introduction
Pose estimation—the process of determining the position and orientation of a camera relative to its environment—is well-understood for single camera-plane pairs. However, scaling this to $m$ cameras and $n$ planes in a unified global coordinate system introduces significant complexity. Two primary hurdles exist: maintaining global consistency across all pairs and handling "missing data," where individual cameras only see a subset of the planes in a scene.
The Challenge of Missing Data
In practical scenarios, such as navigating through multiple rooms, a single camera view rarely captures every planar surface. Standard mathematical tools like factorization typically require a complete data matrix to function. When observations are missing, the resulting "holes" in the data grid make global optimization brittle and prone to failure.
Proposed Methodology
The author proposes a multi-stage pipeline designed to decouple the estimation of rotations and translations, simplifying the overall optimization.
1. Rotation Factorization and Chaining
The core of the approach involves grouping individual rotation matrices into a large global matrix. To address missing entries, the "secret sauce" of this paper is a chaining mechanism. If Camera A does not observe Plane B, the algorithm searches for a "bridge" (e.g., Camera A sees Plane C, and Camera D sees both Plane B and Plane C). By chaining these known transformations, the algorithm populates the missing entries before performing the final factorization.
2. Linear Translation and Bundle Adjustment
Once global rotations are established, translations are solved via a sparse system of linear equations. The entire model—including poses, focal lengths, and lens distortion—is then refined using a non-linear Levenberg-Marquardt optimization to minimize reprojection error.
Experimental Results
The framework was evaluated across three distinct environments:
- Calibration Grid: Achieved angular errors as low as 0.1° and a 3D reconstruction error of 0.1%.
- Indoor Scene: Tested with an 81% missing data rate across 400 images. Despite the extreme sparsity, the algorithm successfully reconstructed the room geometry with an average angular error of approximately 6°.
- Patterned Room: Demonstrated that high-quality feature extraction (using high-contrast patterns) can halve angular errors to roughly 3°, indicating that the algorithm's performance is largely limited by the precision of the underlying image processing.
Conclusion
This research provides a scalable solution for multi-camera, multi-plane pose estimation. By prioritizing rotation consistency and utilizing a graph-based chaining strategy for missing data, the method offers a robust initialization for dense 3D reconstruction. Future work aims to automate the feature extraction process, removing the current requirement for manual interaction.
Summary updated