Automatic Construction of Lane-level HD Maps for Urban Scenes

Yiyang Zhou, Yuichi Takeda, Masayoshi Tomizuka, Wei Zhan

TL;DR

Constructing high-definition (HD) maps for autonomous driving is notoriously expensive and labor-intensive, particularly in messy urban environments. This paper introduces an automated pipeline that combines semantic segmentation, a particle filter-based exploration strategy, and OpenStreetMap (OSM) priors to generate accurate, lane-level HD maps from vehicle sensor data, achieving decimeter-level precision even in complex intersections.

What Problem Does This Paper Address?

For self-driving cars to navigate safely, they rely on HD maps that contain "lane-level" detail—not just where the road is, but exactly where lane boundaries, centerlines, and intersection connections lie.

While highway mapping is relatively mature (thanks to clear markings and simple geometry), urban scenes are a nightmare for automation. They feature frequent lane splits, "invisible" lanes at intersections, potholes, and irregular markings like bus stops or bike lanes. Current methods often rely on tedious manual labeling or struggle with the "3D-ness" of urban roads (like the steep hills of San Francisco). The authors aim to bridge this gap by creating a system that can automatically "trace" the logical and geometric structure of a city's roads.

Methods

The authors propose a multi-stage pipeline that moves from raw pixels to a structured "Directed Cyclic Graph" (DAG) of the road network:

  1. Semantic Scene Understanding: Using a DeepLab-v3+ network, the system segments 2D camera images into 10 classes, including ego-lanes, neighboring lanes, and various road markings.
  2. Topographical BEV Projection: Most systems assume the road is flat when projecting camera images to a Bird’s-Eye-View (BEV). This paper uses an optional LIDAR scan to create a 3D topographical mesh (via Delaunay Triangulation). Projecting semantics onto this "true" ground shape prevents the distortions that usually occur on hills or uneven pavement.
  3. Semantic-Particle Filter Exploration: This is the "aha" part of the paper. Instead of just detecting lines, they release "particles" (simulated vehicles) that "drive" through the BEV semantic map. Particles that stay within drivable areas survive; those that hit boundaries are terminated.
  4. Lane Regression: Surviving particle paths are clustered using DBSCAN to determine the number of lanes. These paths are then smoothed into differentiable curves using piece-wise linear regression and natural splines.
  5. Intersection Inference: Since markings often disappear inside intersections, the authors use OpenStreetMap (OSM) as a "skeleton" to understand which roads connect. They then use second-order Bezier curves to mathematically link incoming and outgoing lanes based on standard traffic rules.

Datasets

The researchers evaluated their system using the UrbanLoco dataset, specifically focusing on highly complex areas:

  • San Francisco (North Beach): Known for steep slopes and irregular 6-way intersections.
  • Downtown Berkeley: Densely urbanized with frequent lane changes.

For the semantic segmentation module, they pre-trained on Cityscapes and fine-tuned on BDD100K, extending the labels to distinguish between multiple neighboring lanes.

Key Insights and Ideas

  • Drivability over Markings: By using a particle filter to explore "drivable semantics" rather than just looking for white lines, the system is robust to scenes where markings are faded, broken, or non-existent.
  • The "Flat Earth" Fallacy: The authors demonstrate that ignoring road topography leads to significant mapping errors. Using a LIDAR-based mesh for BEV projection corrected distorted crosswalks and lane widths, which is critical for centimeter-level HD maps.
  • OSM as a Coarse Prior: Rather than starting from scratch, the system uses the existing (but low-detail) OpenStreetMap database to provide the initial topological "skeleton," which the algorithm then "fleshes out" with high-precision lane data.

Key Results

The system was tested across 47 blocks (approx. 3.5 KM) of urban territory:

  • Accuracy: In Downtown Berkeley, the system achieved a Root Mean Square (RMS) error of 0.24m for lane trajectories and a mean Intersection over Union (mIOU) of 0.79 for lane boundaries.
  • Robustness: In San Francisco, despite the challenging terrain, the RMS error remained low at 0.33m.
  • Intersections: The topological inference (deciding which lane connects to which) reached over 90% precision in urban areas without explicit human supervision.

Limitations and Future Work

The primary limitation identified is occlusion. In heavy traffic, other vehicles block the camera's view of lane markings and curbs, leading to gaps in the semantic map.

To address this, the authors suggest moving toward amodal prediction—essentially training the neural network to "hallucinate" or predict the parts of the road that are hidden behind other cars—and performing more of the semantic processing directly in the BEV domain.

Summary updated

Abstract

High definition (HD) maps have demonstrated their essential roles in enabling full autonomy, especially in complex urban scenarios. As a crucial layer of the HD map, lane-level maps are particularly useful: they contain geometrical and topological information for both lanes and intersections. However, large scale construction of HD maps is limited by tedious human labeling and high maintenance costs, especially for urban scenarios with complicated road structures and irregular markings. This paper proposes an approach based on semantic-particle filter to tackle the automatic lane-level mapping problem in urban scenes. The map skeleton is firstly structured as a directed cyclic graph from online mapping database OpenStreetMap. Our proposed method then performs semantic segmentation on 2D front-view images from ego vehicles and explores the lane semantics on a birds-eye-view domain with true topographical projection. Exploiting OpenStreetMap, we further infer lane topology and reference trajectory at intersections with the aforementioned lane semantics. The proposed algorithm has been tested in densely urbanized areas, and the results demonstrate accurate and robust reconstruction of the lane-level HD map.
BibTeX
@inproceedings{zhou2021automatic,
  author = {Yiyang Zhou and Yuichi Takeda and Masayoshi Tomizuka and Wei Zhan},
  title = {{Automatic Construction of Lane-level HD Maps for Urban Scenes}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2021},
  url = {https://openreview.net/forum?id=17-XfVlmPy},
}

Loading related content...

Figure