Monocular Localization with Vector HD Map (MLVHM): A Low-Cost Method for Commercial IVs

Zhongyang Xiao, Diange Yang, Tuopu Wen, Kun Jiang, Ruidong Yan

TL;DR

This paper introduces MLVHM, a novel, low-cost method for precise vehicle localization using a single camera and vector-based High Definition (HD) maps. By leveraging semantic features and a robust data association technique, MLVHM achieves centimeter-level accuracy without relying on expensive sensors like LiDAR or RTK GNSS, making it suitable for commercial intelligent vehicles.

What Problem Does This Paper Address?

Accurate vehicle localization (knowing precisely where a vehicle is and its orientation) is crucial for many intelligent vehicle (IV) applications, such as autonomous driving and advanced driver-assistance systems. Traditional high-precision localization systems often rely on expensive sensors like RTK GNSS and LiDAR, which are prohibitive for mass-market commercial vehicles. While High Definition (HD) maps are becoming more common, integrating them with low-cost sensors like monocular cameras for accurate, real-time localization remains a challenge. Existing methods often struggle with sensor cost, map data size, and robustness to environmental changes like illumination variations and occlusions. This paper aims to bridge this gap by proposing a cost-effective solution that leverages readily available components: monocular cameras and vector HD maps.

Methods

The core of the MLVHM system is a novel map-matching approach that uses semantic geometric features extracted from monocular camera images and aligns them with a vector HD map.

  • Semantic Feature Extraction: The system identifies and extracts semantic features like lane lines, light poles, and traffic signs from camera images. These features are chosen for their robustness to illumination changes and occlusions. Deep learning models (PSPnet for general segmentation, and a custom encoder-decoder for lane lines) are used for this semantic segmentation.
  • Vector HD Map Utilization: The system uses vector HD maps, which store geometric information (like lane lines and landmark positions) in a compact format, as opposed to dense point clouds.
  • Data Association: A key innovation is an improved RANSAC-based data association method. It randomly samples subsets of potential semantic feature matches, evaluates their quality by projecting map landmarks into the image, and considers the drift from an initial pose estimate to improve robustness against repetitive patterns and ambiguities.
  • Pose Estimation: The camera's 6-DOF pose (position and orientation) is estimated by minimizing re-projection errors between the extracted image features and the corresponding map landmarks using an optimization framework (Levenberg-Marquardt).
  • Frame-to-Frame Motion Fusion: To enhance stability and accuracy, especially in sparse feature environments, the system integrates frame-to-frame motion information obtained from a visual odometry module (based on ORB-SLAM). This is achieved through a sliding window approach that fuses the absolute map-based localization results with the relative motion estimates, effectively smoothing the trajectory and restoring scale lost in monocular visual odometry.

Datasets

The paper does not explicitly mention training datasets. For evaluation, the authors created:

  • Simulated Scenarios: A virtual traffic scenario with varying densities and types of semantic features (lane lines, lamp poles, traffic signs) was generated to test the algorithm under controlled conditions.
  • Real-World Data: Experiments were conducted using two intelligent vehicles equipped with cameras and LiDAR/GNSS systems. The LiDAR data was used to generate ground truth trajectories for comparison. The HD maps were generated from OpenDrive format data.

Key Insights and Ideas

  • Semantic Features for Robustness: The paper strongly advocates for semantic features (lane lines, poles, signs) over traditional visual features (SIFT, ORB) for map matching. These semantic features are inherently more resilient to environmental changes like lighting and weather, which is critical for real-world applications.
  • Vector HD Maps as a Cost-Effective Solution: By focusing on vector HD maps, the authors demonstrate that high-precision localization doesn't necessitate massive point cloud data, making HD maps more practical for commercial deployment.
  • Improved RANSAC for Data Association: The enhanced RANSAC method, which incorporates pose drift into its evaluation, is crucial for handling ambiguous matches in real-world scenarios with repetitive landmarks.
  • Fusion of Absolute and Relative Localization: Combining map-based absolute localization with frame-to-frame visual odometry provides a powerful synergy. The former anchors the system, while the latter smooths the trajectory and improves robustness in challenging areas.
  • Scale Recovery in Monocular SLAM: The sliding window fusion technique is shown to effectively recover the scale lost in monocular visual odometry, a common limitation of such systems.

Key Results

  • High Accuracy: The MLVHM algorithm achieved a Root Mean Square Error (RMSE) of 24 cm in typical traffic scenarios with no cumulative error.
  • Competitive Performance: The system achieved comparable or better localization accuracy than existing map-matching algorithms, even with lower-cost sensor configurations and lighter HD maps.
  • Real-time Performance: After fusing frame-to-frame constraints, the system achieved an average output delay of 0.059 seconds, demonstrating its suitability for real-time applications.
  • Improved Smoothness: The frame-to-frame motion fusion significantly improved trajectory smoothness, reducing outliers and making it more favorable for motion planning.
  • Feature Density Impact: Localization accuracy is sensitive to feature density and distribution, with denser features generally leading to better performance.

Limitations and Future Work

  • Limitations:
    • The paper acknowledges that the localization accuracy is sensitive to feature density and distribution.
  • While semantic segmentation is robust, its computational cost can be a bottleneck, leading to an average output delay of 0.515s before fusion.
  • The initial version of the work had limitations in data association, which were improved in this paper.
  • Future Work:
    • The authors plan to incorporate road curves and other complex road patterns into the matching error functions to further enhance adaptability.
  • They intend to include other low-cost localization sensors, such as GNSS, to meet the evolving needs of intelligent vehicles.
  • Further research will focus on improving the real-time performance, particularly by optimizing the semantic segmentation module.

Summary updated

Abstract

Real-time vehicle localization (i.e., position and orientation estimation in the world coordinate system) with high accuracy is the fundamental function of an intelligent vehicle (IV) system. In the process of commercialization of IVs, many car manufacturers attempt to avoid high-cost sensor systems (e.g., RTK GNSS and LiDAR) in favor of low-cost optical sensors such as cameras. The same cost-saving strategy also gives rise to an increasing number of vehicles equipped with High Definition (HD) maps. Rooted upon these existing technologies, this article presents the concept of Monocular Localization with Vector HD Map (MLVHM), a novel camera-based map-matching method that efficiently aligns semantic-level geometric features in-camera acquired frames against the vector HD map in order to achieve high-precision vehicle absolute localization with minimal cost. The semantic features are delicately chosen for the ease of map vector alignment as well as for the resiliency against occlusion and fluctuation in illumination. The effective data association method in MLVHM serves as the basis for the camera position estimation by minimizing feature re-projection errors, and the frame-to-frame motion fusion is further introduced for reliable localization results. Experiments have shown that MLVHM can achieve high-precision vehicle localization with an RMSE of 24 cm with no cumulative error. In addition, we use low-cost on-board sensors and light-weight HD maps to achieve or even exceed the accuracy of existing map-matching algorithms.
BibTeX
@inproceedings{xiao2020monocular,
  author = {Zhongyang Xiao and Diange Yang and Tuopu Wen and Kun Jiang and Ruidong Yan},
  title = {{Monocular Localization with Vector HD Map (MLVHM): A Low-Cost Method for Commercial IVs}},
  booktitle = {Sensors},
  year = {2020},
  url = {https://openreview.net/forum?id=iYXLYFTewBp},
}
Figure