MCGMapper: Light-Weight Incremental Structure from Motion and Visual Localization With Planar Markers and Camera Groups

Yusen Xie, Zhenmin Huang, Kai Chen, Lei Zhu, Jun Ma

TL;DR

This paper introduces MCGMapper, a novel framework for robust 3D reconstruction and visual localization in challenging indoor environments using planar markers. It addresses limitations of existing methods by effectively handling varying marker sizes and utilizing camera groups for improved scene coverage, leading to more accurate and faster map building.

What Problem Does This Paper Address?

Structure from Motion (SfM) and visual localization are crucial for robotics and augmented reality, especially in indoor, texture-less, or repetitive environments like warehouses. Traditional SfM methods struggle in these scenarios due to difficulties in extracting reliable features and establishing correspondences. While visual markers offer a solution with their distinct features, existing marker-based SfM methods suffer from slow speeds, convergence issues, and a strict assumption of uniform marker size. Furthermore, they often rely on single monocular cameras, leading to limited field-of-view and potential map discontinuities. This paper aims to overcome these limitations by proposing a more robust, efficient, and flexible marker-based SfM framework.

Methods

The proposed MCGMapper framework is an incremental SfM approach that combines a front-end using Perspective-n-Point (PnP) for initial pose estimation with a customized back-end bundle adjustment.

  • Marker Bundle Adjustment: A novel bundle adjustment is derived to handle markers of different sizes and an arbitrary number of cameras within camera groups. It incorporates orthogonality constraints of marker corners and known camera extrinsics as prior information.
  • Camera Groups Bundle Adjustment: This component leverages the known transformations between cameras in a group to improve pose estimation, treating the group as a single entity.
  • Incremental SfM Framework: The system operates in three stages:
  • Map Initialization: A random frame is selected, its pose is fixed as the origin, and initial marker poses are estimated using PnP.
  • Global Visual Localization: For new frames, a local factor graph is constructed using recognized markers. The camera pose is optimized by averaging errors from co-viewed markers (from the existing map) and incorporating non-co-viewed markers through PnP.
  • Incremental Map Update: New observations are used to update the global map. Non-co-viewed markers are added to the map, and then a global optimization is performed on the entire map.
  • Observation Weighting: A method is introduced to measure observation weights, considering factors like marker distance, viewing angle, and rotation solution ambiguity from IPPE, to improve the robustness of the information matrix used in optimization.

Datasets

The authors introduce a new synthetic dataset generated using Blender [20] that includes various indoor scenes (rooms, corridors, warehouses, swimming pools, calibration rooms) with ground-truth pose labels. This dataset features scenarios with both uniform and varying marker sizes, and supports monocular and camera group (CG-60, CG-120) capture configurations. They also evaluate their method on public datasets like SPM-SLAM [6] and Degol [7].

Key Insights and Ideas

  • Handling Diverse Marker Sizes: The ability to handle markers of different sizes is a significant advancement, making the system more adaptable to real-world scenarios where uniform marker placement isn't always feasible.
  • Camera Groups for Enhanced Coverage: Utilizing camera groups (CG) demonstrably improves reconstruction accuracy and robustness, especially in large or complex scenes, by increasing the field-of-view and providing more observational data.
  • Optimized Bundle Adjustment: The customized bundle adjustment, incorporating marker and camera group constraints, leads to more accurate and faster convergence compared to general methods.
  • Robust Localization with Weighted Observations: The proposed observation weighting scheme helps to mitigate ambiguities and errors in marker pose estimation, leading to more reliable global localization.
  • Synthetic Data for Benchmarking: The creation of a synthetic dataset with ground truth is crucial for quantitative evaluation, addressing a scarcity in existing marker-based SfM research.

Key Results

  • Accuracy: MCGMapper achieves state-of-the-art accuracy on public datasets and significantly outperforms existing methods on their proposed synthetic datasets, particularly in large and complex scenes.
  • Speed: The framework is described as "light-weight" and "fast," with experiments showing processing times for large datasets within minutes. For instance, a dataset with 896 images was processed in under 15 minutes.
  • Robustness: The method demonstrates robustness in challenging scenarios, including texture-less environments and those with varying marker sizes. Camera group configurations (CG-60, CG-120) show marked improvements over monocular setups.
  • Quantitative Metrics: Tables in the paper show lower RMSE for Absolute Trajectory Error (ATE) for their method compared to baselines like PytagMapper, especially when using camera groups.

Limitations and Future Work

  • Limitations:
    • The paper acknowledges that in some atypical scenarios, such as oblique observation angles or long distances, determining the correct marker pose can still be challenging, although their weighting scheme helps.
  • While the method is efficient, the computational cost can still be significant for extremely large datasets, as indicated by processing times.
  • The paper mentions that for vast scenarios, some algorithms (like Degol) fail to reconstruct, implying that MCGMapper, while better, might still face challenges in extremely large-scale environments.
  • Future Work: The authors suggest that their proposed dataset and framework provide a valuable resource for future research in marker-based SfM. The paper doesn't explicitly detail future research directions, but the robustness and flexibility of MCGMapper suggest potential applications in areas like autonomous navigation, augmented reality, and industrial inspection.

Summary updated

BibTeX
@inproceedings{xie2024mcgmapper,
  author = {Yusen Xie and Zhenmin Huang and Kai Chen and Lei Zhu and Jun Ma},
  title = {{MCGMapper: Light-Weight Incremental Structure from Motion and Visual Localization With Planar Markers and Camera Groups}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2024},
  doi = {10.1109/iros58592.2024.10802612},
}
Figure