Exploiting Sparse Semantic HD Maps for Self-Driving Vehicle Localization

Wei-Chiu Ma, Raquel Urtasun, Ignacio Tartavull, Ioan Andrei Bârsan, Shenlong Wang, Min Bai, Gellért Máttyus, Namdar Homayounfar, Shrinidhi Kowshika Lakshmikanth, Andrei Pokrovsky

Abstract

High-definition (HD) maps are critical for autonomous vehicle localization but typically rely on dense, storage-intensive LiDAR data. This paper proposes a lightweight alternative using sparse semantic features—lane graphs and traffic signs—to achieve centimeter-level accuracy. By formulating localization as a Bayesian histogram filter, the system reduces map storage requirements by 99.7% (to ~0.55 MiB/km²) compared to traditional dense maps. Evaluations on a 312 km highway dataset demonstrate a median lateral error of 0.05m and a median longitudinal error of 1.12m, proving that sparse semantics are sufficient for safe autonomous navigation.

Introduction

Standard HD maps built from dense LiDAR point clouds are highly accurate but difficult to scale due to their massive storage requirements (often gigabytes per square kilometer). This makes real-time map updates over wireless networks impractical. Furthermore, while GPS and Inertial Measurement Units (IMUs) provide basic positioning, they suffer from multi-meter errors and temporal drift. This work introduces a "middle ground" system: a localization framework that maintains the precision of dense map methods while utilizing a map as lightweight as a standard navigation map.

Methodology

The system treats localization as a Bayesian filtering problem, specifically employing a histogram filter to integrate multi-modal sensor data with a sparse semantic map.

System architecture overview showing the integration of sensor data, semantic perception, and lightweight maps into a posterior probability distribution.

Sparse Semantic Mapping

The lightweight map consists of two primary semantic layers:

  1. Lane Graphs: Polylines representing the road's lane structure.
  2. BEV Sign Map: 3D locations of traffic signs projected into a Bird's-Eye View (BEV).

Perception and Inference

A multi-sensor CNN processes LiDAR and camera data to detect lanes, outputting a distance transform. Simultaneously, a semantic segmentation network (PSPNet) identifies traffic signs in camera frames, which are then projected into 3D space using LiDAR. To maintain real-time performance (~7Hz), the system utilizes FFT-convolutions to match live sensor observations against the stored semantic map.

Complementary Constraints

A core insight of this approach is the exploitation of complementary geometric constraints. Lanes provide excellent lateral (side-to-side) constraints but are longitudinally ambiguous in repetitive highway environments. Traffic signs, though sparse, provide the necessary longitudinal (forward-back) anchors to correct IMU drift.

Visualization of observation probabilities for lanes, signs, and GPS, and their fusion into a final posterior pose estimate.

Experimental Evaluation

The researchers evaluated the system on a 312 km dataset of North American highways, encompassing various lighting conditions and times of day. Ground truth was established using high-precision offline Graph-SLAM.

Key Results

  • Accuracy: The system achieved a median lateral error of 0.05m and a median longitudinal error of 1.12m.
  • Efficiency: Map storage was reduced from 1.4 GiB/km² to just 0.55 MiB/km², a 99.7% reduction.
  • Robustness: The histogram filter allowed the vehicle to handle uncertainty gracefully, maintaining smooth trajectories even in stretches of road with sparse signage where geometric matchers typically fail.

Conclusion

This work demonstrates that sparse semantic features can replace dense geometric maps for high-precision highway localization. While highly effective on highways, future work will focus on extending this approach to complex urban environments by incorporating additional semantic features such as poles, curbs, and building facades to handle increased environmental complexity and occlusion.

Summary updated

Abstract

In this paper we propose a novel semantic localization algorithm that exploits multiple sensors and has precision on the order of a few centimeters. Our approach does not require detailed knowledge about the appearance of the world, and our maps require orders of magnitude less storage than maps utilized by traditional geometry- and LiDAR intensity-based localizers. This is important as self-driving cars need to operate in large environments. Towards this goal, we formulate the problem in a Bayesian filtering framework, and exploit lanes, traffic signs, as well as vehicle dynamics to localize robustly with respect to a sparse semantic map. We validate the effectiveness of our method on a new highway dataset consisting of 312km of roads. Our experiments show that the proposed approach is able to achieve 0.05m lateral accuracy and 1.12m longitudinal accuracy on average while taking up only 0.3% of the storage required by previous LiDAR intensity-based approaches.
BibTeX
@inproceedings{ma2019exploiting,
  author = {Wei-Chiu Ma and Raquel Urtasun and Ignacio Tartavull and Ioan Andrei Bârsan and Shenlong Wang and Min Bai and Gellért Máttyus and Namdar Homayounfar and Shrinidhi Kowshika Lakshmikanth and Andrei Pokrovsky},
  title = {{Exploiting Sparse Semantic HD Maps for Self-Driving Vehicle Localization}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2019},
  url = {https://openreview.net/forum?id=w3mSri_KwX},
}
Figure