Matrix-Free Shared Intrinsics Bundle Adjustment

Daniel SafariCVPR 2025 · DOI

Abstract

Bundle Adjustment (BA) is a fundamental optimization step in 3D reconstruction, yet a significant gap exists between academic benchmarks and industrial requirements. While standard benchmarks often assume unique camera parameters for every image, real-world applications in robotics and augmented reality (AR) typically involve shared intrinsics (SI-BA). This paper introduces a specialized solver for SI-BA that leverages matrix-free computation and numerical safeguards to enable stable single-precision (32-bit) optimization. The proposed approach achieves up to an 8x speedup and a 10x reduction in memory consumption compared to industry-standard solvers like Ceres, without compromising convergence accuracy.

Introduction

Bundle Adjustment serves as the "engine room" of 3D reconstruction, fine-tuning camera poses and 3D landmarks to minimize reprojection error. However, general-purpose solvers often fail to exploit the specific mathematical structure of Shared Intrinsics Bundle Adjustment (SI-BA). In industrial contexts—such as a drone capturing a flight sequence or a user navigating with an AR headset—multiple images are captured by the same physical sensor, sharing identical intrinsic parameters. This paper addresses the need for a solver optimized for this common scenario, focusing on computational efficiency and memory footprint.

Methodology

Matrix-Free Computation

Traditional BA solvers precompute and store massive Hessian matrices to represent the relationships between cameras and landmarks. However, modern hardware is often memory-bound rather than compute-bound. The proposed solver adopts a "matrix-free" approach, calculating these values on the fly during optimization iterations. By avoiding heavy memory fetches and keeping data within high-speed CPU/GPU caches, the solver significantly reduces the bottleneck associated with data movement.

Single-Precision Synthesis

To further optimize performance, the solver utilizes 32-bit single-precision math. While single-precision is faster and uses half the memory of 64-bit double-precision, it is traditionally avoided due to numerical instability. This paper introduces three key mitigations to ensure stability:

  1. Origin Shifting: Centering the coordinate system on the scene to prevent catastrophic cancellation.
  2. Balanced Scaling: A numerical scaling factor that balances the influence of camera translation and rotation.
  3. Low-Parallax Handling: Efficient identification and management of distant 3D points that could otherwise destabilize the optimization.
Comparison of condition numbers (κ) for different scaling methods, demonstrating the stability of the proposed approach.

As shown in the figure above, the proposed scaling method significantly improves the condition number compared to unscaled or standard Jacobi-scaled systems, making single-precision optimization viable.

Experimental Evaluation

Datasets

To evaluate the solver, the author curated a suite of SI-BA models derived from prominent community datasets, ranging from high-quality reconstructions to large-scale AR maps.

Statistics of the datasets used for evaluation, highlighting the scale of observations and landmarks.

Performance Results

The proposed solver was compared against Google's Ceres solver across these datasets. Key findings include:

  • Speed: The CPU-based implementation is approximately 8x faster than Ceres, while the GPU (CUDA) version achieves up to a 20x speedup on specific datasets.
  • Memory Efficiency: On the large-scale LaMAR dataset, the proposed solver reduced memory requirements from 8.7 GB (Ceres) to just 1.6 GB.
  • Accuracy: Despite the use of single-precision math, the solver achieved convergence levels identical to double-precision alternatives.

Conclusion

This work demonstrates that by tailoring solvers to the shared intrinsics structure and prioritizing compute over memory storage, 3D reconstruction can be made significantly more efficient. The introduction of numerical safeguards proves that single-precision math is a viable and high-performance alternative for industrial-scale Bundle Adjustment. Future work will focus on further optimizing the camera-landmark relationship blocks to handle even more complex multi-camera configurations.

Summary updated

BibTeX
@inproceedings{safari2025matrix,
  author = {Daniel Safari},
  title = {{Matrix-Free Shared Intrinsics Bundle Adjustment}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2025},
  doi = {10.1109/cvpr52734.2025.02516},
}
Figure