New Monte Carlo Localization Using Deep Initialization: A Three-Dimensional LiDAR and a Camera Fusion Approach

HyungGi Jo, Euntai KimIEEE Access 2020 · DOI

Introduction

Global localization—the ability of a robot to determine its position on a map without prior knowledge of its starting point—is a fundamental requirement for autonomous navigation. While Global Positioning Systems (GPS) are commonly used, they are unreliable in "GPS-shaded" environments such as indoor warehouses, parking garages, and dense urban canyons.

Robotics researchers frequently employ Monte Carlo Localization (MCL), a particle filter-based method that estimates a robot's pose by maintaining a set of hypothetical positions (particles) and refining them based on sensor data. However, standard MCL faces two significant challenges: the "convergence lag," where a uniform distribution of particles takes considerable time and movement to converge, and the "kidnapped robot problem," where the system fails to recover if the robot is moved or tracking is lost. This paper introduces a "Deep Initialization" approach to overcome these hurdles by fusing monocular camera pose regression with 3D LiDAR-based filtering.

Methodology

The proposed system architecture integrates deep learning with traditional probabilistic filtering to achieve rapid and robust localization.

System architecture of the proposed Deep Initialization MCL framework.

Deep Initialization via Monocular Camera

Instead of initializing particles uniformly across the map, the system uses a monocular camera to generate a high-quality initial estimate. A Convolutional Neural Network (CNN), based on the PoseNet architecture with a GoogLeNet backbone, regresses a single image into an absolute 6-DOF pose. While this estimate lacks the precision required for fine navigation, it significantly narrows the search space for the particle filter.

Manifold-Based Sampling

To convert the CNN's single pose estimate into a particle set, the authors employ sampling in the tangent space (Lie algebra $se(3)$) of the $SE(3)$ manifold. This approach ensures that the particle distribution represents valid 3D rigid motions, avoiding mathematical issues like gimbal lock associated with Euler angles.

3D LiDAR Refinement and Failure Detection

Once initialized, a 3D LiDAR-based MCL takes over, comparing live scans to a pre-built 3D point cloud map to achieve high-precision pose estimation. The system continuously monitors for localization failures by comparing the LiDAR-based pose with the camera-based CNN output. If a significant divergence is detected, a fresh "Deep Initialization" is triggered to reset the particles and recover from the failure.

Experimental Setup

The system was evaluated using a custom robot platform equipped with a 3D LiDAR and a monocular camera.

The experimental robot platform showing the sensor configuration.

Testing was conducted in two complex urban environments in South Korea: the Engineering Building at Yonsei University (YU_ENGR) and the Innovation Building at Korea University (KU_INNO). These sites featured transitions between indoor and outdoor spaces. Approximately 9,000 to 10,000 images per site were used to train the pose regression network, and high-resolution 3D maps were generated using LiDAR-based SLAM.

Results and Discussion

The proposed method demonstrated substantial improvements over traditional 2D and 3D MCL baselines across several metrics:

  • Convergence Speed: The First Correct Localization Time (FCLT) was reduced from approximately 1,500–2,000 steps in standard 3D-MCL to just 101–138 steps.
  • Reliability: In the YU_ENGR environment, the Correct Localization Rate (CLR) increased from ~24% to over 90%.
  • Accuracy: The Root-Mean-Square Error (RMSE) for the final pose was reduced to 0.56m in YU_ENGR and 0.88m in KU_INNO.

By utilizing a targeted initialization, the system requires significantly fewer particles (500 compared to up to 30,000 in traditional methods), leading to greater computational efficiency.

Conclusion

This research presents a robust fusion approach for global localization in GPS-denied environments. By combining the rapid "guess" of a deep learning-based camera pose regressor with the precision of 3D LiDAR-based MCL, the system achieves near-instantaneous convergence and effective recovery from the kidnapped robot problem. Future work will focus on improving the generalization of the pose regression network to new environments and enhancing robustness to dynamic visual changes.

Summary updated

BibTeX
@article{jo2020new,
  author = {HyungGi Jo and Euntai Kim},
  title = {{New Monte Carlo Localization Using Deep Initialization: A Three-Dimensional LiDAR and a Camera Fusion Approach}},
  journal = {IEEE Access},
  year = {2020},
  doi = {10.1109/access.2020.2988464},
}
Figure