Automatic Building and Labeling of HD Maps with Deep Learning

Mahdi Elhousni, Yecheng Lyu, Ziming Zhang, Xinming Huang

Abstract: In a world where autonomous driving cars are becoming increasingly more common, creating an adequate infrastructure for this new technology is essential. This includes building and labeling high-definition (HD) maps accurately and efficiently. Today, the process of creating HD maps requires a lot of human input, which takes time and is prone to errors. In this paper, we propose a novel method capable of generating labelled HD maps from raw sensor data. We implemented and tested our methods on several urban scenarios using data collected from our test vehicle. The results show that the proposed deep learning based method can produce highly accurate HD maps. This approach speeds up the process of building and labeling HD maps, which can make meaningful contribution to the deployment of autonomous vehicles.

Introduction

High-Definition (HD) maps are essential for autonomous vehicles, providing a "prior" of the environment that helps with precise localization and path planning. However, creating these maps is currently a massive bottleneck. While LiDAR is excellent at capturing 3D geometry, raw point clouds lack semantic information—lanes and road boundaries are often invisible or appear as noisy intensity data. Consequently, most HD maps are labeled manually by humans, a process that is slow, expensive, and prone to error. This research aims to bridge this gap by using Deep Neural Networks (DNNs) to identify semantics in camera images and automatically project those labels onto the 3D LiDAR map.

Methodology

The proposed multi-stage pipeline separates the task into three primary components: 3D Reconstruction, Road Mapping, and Lane Mapping.

Figure 1: The overall automated pipeline for HD map generation, integrating NDT-based reconstruction with road and lane mapping modules.

3D Reconstruction

The system uses the 3D Normal Distribution Transform (3D NDT) algorithm to align successive LiDAR scans into a consistent global point cloud, providing the geometric foundation for the map.

Road Mapping and Curb Detection

Road detection begins with a Fully Convolutional Network (FCN) segmenting the road in camera images. However, visual segmentation often "bleeds" over edges. To refine this, the system employs Otsu’s method on elevation histograms. Since curbs are slightly higher than roads, the elevation data follows a bimodal distribution. This allows the system to mathematically isolate the curb and prune the road mask for high precision. Finally, a Concave Hull (CH) algorithm generates a clean polygon representing the drivable area.

Figure 5: Elevation histogram used for curb detection. The bimodal distribution allows for mathematical isolation of the curb from the road surface.

Lane Mapping

Lane detection is handled by LaneNET, which identifies multiple lane instances in the 2D image. These pixels are projected into 3D space. To handle sensor noise, the system applies quadratic and logarithmic curve fitting to smooth the lane trajectories. In cases where the camera misses a lane due to lighting or occlusion, the system uses the detected curb positions and standard lane widths to infer and generate the missing markings.

Key Insights and Results

A critical innovation in this pipeline is using geometry to refine semantics. While cameras may struggle with shadows at road boundaries, LiDAR detects the physical "step" of the curb. Using Otsu’s thresholding on 3D elevation data provides a much sharper boundary than vision alone. Furthermore, by using a monocular camera for detection and LiDAR only for spatial positioning, the system bypasses the limitations of low-resolution LiDAR intensity data.

Experimental results across five urban scenarios (straightRoad, curvedRoad, mergeLane, intersection, and highway) demonstrate high accuracy:

  • Road Accuracy: In the straightRoad scenario, curb detection refinement reduced the mapped road area error from 0.232 m² to just 0.008 m².
  • Lane Precision: Translation error for automatically labeled lanes generally remained between 0.3m and 0.9m, even in complex intersections.

Conclusion

This automated approach demonstrates that HD maps can be built offline using recorded sensor logs, potentially replacing hundreds of hours of manual labor. While the current system relies on distinct curbs—which may limit its use on rural or unstructured roads—it provides a robust framework for urban environments. Future work will focus on incorporating additional semantic features such as traffic signs and lights to create a comprehensive HD mapping solution.

Summary updated

Abstract

In a world where autonomous driving cars are becoming increasingly more common, creating an adequate infrastructure for this new technology is essential. This includes building and labeling high-definition (HD) maps accurately and efficiently. Today, the process of creating HD maps requires a lot of human input, which takes time and is prone to errors. In this paper, we propose a novel method capable of generating labelled HD maps from raw sensor data. We implemented and tested our methods on several urban scenarios using data collected from our test vehicle. The results show that the proposed deep learning based method can produce highly accurate HD maps. This approach speeds up the process of building and labeling HD maps, which can make meaningful contribution to the deployment of autonomous vehicles.
BibTeX
@inproceedings{elhousni2020automatic,
  author = {Mahdi Elhousni and Yecheng Lyu and Ziming Zhang and Xinming Huang},
  title = {{Automatic Building and Labeling of HD Maps with Deep Learning}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2020},
  url = {https://openreview.net/forum?id=RO8XFCGQzO},
}
Figure