Abstract
Precise localization is a cornerstone of autonomous driving. While LiDAR sensors provide rich 3D maps, their cost and data volume present significant challenges for mass-market vehicles. Monocular cameras offer a cost-effective alternative but struggle with cross-modal matching to 3D point clouds. This paper presents LHMap-loc, a high-performance pipeline that localizes a monocular camera within a LiDAR map by compressing point clouds into "LiDAR Heat Maps" (LHMap). By utilizing an optical flow-based pose regression network, LHMap-loc achieves state-of-the-art accuracy, maintains a 77 FPS processing speed, and reduces map storage requirements by 80%.
Introduction
The "where am I?" problem in autonomous driving is typically solved using expensive LiDAR sensors or high-definition (HD) maps. Monocular localization—matching a 2D image to a 3D map—is desirable for its low cost but faces two primary hurdles: the massive storage and computational overhead of raw LiDAR point clouds, and the inherent difficulty of cross-modal feature matching between 2D pixels and 3D points.
LHMap-loc addresses these issues by identifying the most informative points in a LiDAR map for visual localization, creating a sparse yet effective representation that bridges the dimensionality gap.
Methodology
The LHMap-loc framework consists of an offline map generation stage and an online localization stage.
Offline LHMap Generation
Instead of keeping every point in a LiDAR map, the authors train a network to identify which points are most "useful" for visual localization based on visibility and distinctiveness. The network generates a "heat value" for each point. By selecting only the top 20% of points with the highest heat values, the system creates a LiDAR Heat Map (LHMap). This effectively compresses the original map by 80% while retaining the structural "skeleton" needed for matching.
Online Pose Regression
During operation, the system regresses the 6-Degree-of-Freedom (6-DoF) pose in real-time using a two-step approach:
- Projection and Flow: The LHMap is projected onto the current camera view. A deep network, inspired by optical flow architectures (PWCNet), calculates the displacement between the projected map and the actual RGB image.
- Spatial Attention: A weighting mechanism helps the network focus on reliable environmental features while ignoring noise or dynamic objects like other vehicles and pedestrians.
- End-to-End Learning: The network directly regresses the camera's position and orientation (quaternions).
Experiments and Results
The researchers evaluated their method on three major benchmarks: the KITTI Odometry Dataset, the Argoverse Dataset, and a custom SJTU Dataset collected in challenging real-world scenarios like lakeside paths and heavy foliage.
Key Results
- Accuracy: On the KITTI dataset, LHMap-loc achieved a translation error of just 0.03 meters and a rotation error of 0.33° after three iterations.
- Efficiency: The system processes at ~77 FPS (13.4ms per frame), making it more than fast enough for real-time autonomous navigation.
- Reliability: While other methods showed failure rates on Argoverse, LHMap-loc maintained a 0% failure rate, demonstrating superior robustness.
- Compression: The map size for KITTI sequence 00 was reduced from 511MB to 109MB without losing localization performance.
Discussion and Conclusion
LHMap-loc proves that "quality over quantity" is effective in mapping; 20% of carefully selected data can yield higher accuracy than 100% of raw data. Furthermore, using optical flow techniques to bridge the gap between 3D projections and 2D images is highly effective for cross-modal alignment. Future work will focus on handling long-term temporal changes (e.g., seasonal variations) and reducing the dependency on a rough initial pose for the regression process.
Summary updated