Hybrid Camera Pose Estimation with Online Partitioning

Xinyi Li, Haibin Ling

Abstract

This paper presents a hybrid real-time camera pose estimation framework with a novel partitioning scheme and introduces motion averaging to monocular Simultaneous Localization and Mapping (SLAM) systems. Breaking through the limitations of fixed-size temporal partitioning in many conventional SLAM pipelines, our approach significantly improves the accuracy of local bundle adjustment by gathering spatially-strongly-connected cameras into each block. With the dynamic initialization using intermediate computation values, we improve the Levenberg-Marquardt solver to further enhance the efficiency of the local optimization. Moreover, the dense data association between blocks by our co-visibility-based partitioning enables us to explore and implement motion averaging to efficiently align the blocks globally, updating camera motion estimations on-the-fly. Experiments on benchmarks convincingly demonstrate the practicality and robustness of our proposed approach by significantly outperforming conventional approaches.

Introduction: The Complexity of Bundle Adjustment

At the heart of most Simultaneous Localization and Mapping (SLAM) and Structure-from-Motion (SfM) systems is Bundle Adjustment (BA)—an optimization process that jointly refines 3D landmark coordinates and camera poses. However, BA is notoriously computationally expensive, scaling cubically with the number of images.

To handle this in real-time, conventional SLAM systems typically partition incoming video frames into fixed-size temporal blocks (e.g., optimizing 30 frames at a time). This naive "chunking" ignores spatial relationships, often leading to poor initialization, slow convergence, and accumulated drift over long trajectories. While offline systems solve this using hierarchical divide-and-conquer approaches, those require the entire video to be available in advance. This research tackles the challenge of bringing the accuracy and efficiency of divide-and-conquer BA into a real-time, online SLAM setting.

Input video frames and feature correspondences used for pose estimation.

Methodology: A Hybrid Pipeline

The proposed framework fundamentally changes how frames are grouped and optimized by moving from temporal windows to spatial co-visibility. The pipeline consists of four primary stages: input processing, online partitioning, local refinement, and global alignment.

The proposed hybrid pipeline: from input and partitioning to local refinement and global alignment.

1. Online Co-visibility Partitioning

Instead of grouping frames strictly by time, the system dynamically builds blocks based on local and global co-visibility. If a new frame shares enough 3D points with the current block, it is added. Crucially, cameras from previous blocks can also be pulled into the current block if they share overlapping views, creating spatially-strongly-connected clusters. This ensures that the local BA has enough overlapping data to converge accurately.

2. Self-Adaptive Local Optimization

To optimize these blocks, the authors introduce a modified Levenberg-Marquardt (L-M) solver. Because cameras are shared across blocks, the system can initialize the optimization of a new block using the intermediate pose calculations from the previous block. This "warm start" is broadcasted across the block via a Maximum Spanning Forest, ensuring the solver starts very close to the optimal solution. This prevents the local optimizer from getting stuck in local minima, a common problem in standard BA.

3. Global Alignment via Motion Averaging

Instead of running a massive, slow global BA on all point-camera correspondences to fix drift, the system uses single rotation averaging. Because the dynamic partitioning naturally forces blocks to share camera poses, the system can globally align the entire trajectory simply by averaging the shared camera poses across different blocks on-the-fly. This approach bypasses the need for expensive explicit loop-closure detection, as global alignment happens organically through the co-visibility links.

Experimental Evaluation

The framework was evaluated on two standard benchmarks: the TUM RGB-D dataset (indoor scenes with rotation-only motions) and the KITTI Odometry dataset (large-scale outdoor trajectories).

Key Results

  • Massive Speedup: The global alignment step is 60 to 150 times faster than conventional point-camera global BA, maintaining near-constant run-time even as the number of frames increases.
  • Efficiency: Thanks to the smart initialization and self-adaptive L-M solver, the local BA requires significantly fewer iterations to converge compared to fixed-size partitioning.
  • Robustness: The method outperforms state-of-the-art systems like ORB-SLAM and DVO-SLAM. On the KITTI dataset, it successfully maintained tracking on sequences where conventional systems failed entirely and showed drastically reduced drift on long-range trajectories.

Discussion and Limitations

While highly effective, the system has specific operational constraints. In environments lacking distinct visual features or during extremely fast motion, the system struggles to find co-visibility, causing the dynamic partitioning to degrade back to standard fixed-size temporal partitioning. Additionally, the system's accuracy is tied to feature density; while it works with sparse features like ORB, it achieves peak performance with denser extractors like SIFT. Finally, to prevent computational bloat in highly dense scenarios (e.g., lingering in one room), the authors implemented a hard upper bound on the number of overlapping cameras allowed per block.

In conclusion, by prioritizing spatial co-visibility over temporal sequence and utilizing motion averaging for global consistency, this framework provides a practical and robust solution for real-time monocular SLAM in diverse environments.

Summary updated

Abstract

This paper presents a hybrid real-time camera pose estimation framework with a novel partitioning scheme and introduces motion averaging to monocular Simultaneous Localization and Mapping (SLAM) systems. Breaking through the limitations of fixed-size temporal partitioning in many conventional SLAM pipelines, our approach significantly improves the accuracy of local bundle adjustment by gathering spatially-strongly-connected cameras into each block. With the dynamic initialization using intermediate computation values, \XL{we improve the Levenberg-Marquardt solver to further enhance the efficiency of the local optimization.} Moreover, the dense data association between blocks by our co-visibility-based partitioning enables us to explore and implement motion averaging to efficiently align the blocks globally, updating camera motion estimations on-the-fly. Experiments on benchmarks convincingly demonstrate the practicality and robustness of our proposed approach by significantly outperforming conventional approaches.
BibTeX
@article{li2020hybrid,
  author = {Xinyi Li and Haibin Ling},
  title = {{Hybrid Camera Pose Estimation with Online Partitioning}},
  journal = {IEEE Robotics Autom. Lett.},
  year = {2020},
  url = {https://openreview.net/forum?id=mP7N-0sHhjH},
}
Figure