Lane-Level Street Map Extraction From Aerial Imagery

Songtao He, Hari BalakrishnanWACV 2022

TL;DR

This paper introduces a novel two-stage pipeline for automatically extracting detailed, routable lane-level street maps from aerial imagery. This is a significant step beyond existing road-level mapping, enabling more precise navigation and autonomous vehicle applications by capturing lane geometry and direction.

What Problem Does This Paper Address?

Creating and maintaining detailed digital maps, especially those with lane-level information, is a labor-intensive and expensive process. Existing methods for extracting road networks from aerial imagery typically operate at a coarser "road level," failing to capture the crucial details like individual lanes, lane merging/diverging, and turning restrictions necessary for advanced applications like lane-to-lane navigation or precise autonomous vehicle routing. While vehicle-mounted sensors can provide this detail, they are costly to deploy at scale. This paper aims to bridge this gap by developing an automated system that can extract these high-fidelity lane-level maps from readily available aerial imagery.

Methods

The authors propose a two-stage mapping pipeline:

  • Sub-task 1: Lane Extraction at Non-Intersection Areas: This stage focuses on extracting lane geometry and direction in areas without intersections. They employ a convolutional neural network (CNN) with a UNet-like architecture, incorporating a ResNet encoder and dilated convolutional layers for improved feature extraction. This model simultaneously predicts a lane segmentation map (identifying lane pixels) and a lane direction map (encoding lane orientation). The outputs are then combined to form a directed lane graph for these planar regions.
  • Sub-task 2: Turning Lane Extraction at Intersections: Intersections present a challenge because the lane graph is no longer planar. Instead of attempting to extract the entire intersection graph at once, they break it down:
  • Terminal Node Identification: They identify the endpoints of lanes at intersections.
  • Candidate Turning Lane Enumeration: They enumerate all possible pairs of terminal nodes that could be connected by a turning lane, considering pairs within a certain distance threshold.
  • Turning Lane Validation: A binary classification model, aided by "reachable lane segmentation" (lanes reachable from a terminal node), is used to determine if a candidate turning lane is valid.
  • Turning Lane Extraction: For validated candidates, they extract the turning lane geometry using a segmentation approach and then convert it into a lane vector. Finally, the extracted turning lanes are merged with the non-intersection lane graph to form a complete, routable lane graph.

Datasets

The authors created their own dataset for evaluation, covering approximately 400 km of lanes across four US cities: Miami, Boston, Seattle, and Phoenix. The aerial imagery was collected from Mapbox and resampled to a ground sampling distance (GSD) of 0.125 meters/pixel. Lane graph labels were adapted from the Argoverse dataset for Miami and manually annotated for the other three cities. The dataset is organized into 35 tiles, with 24 used for training and 11 for testing.

Key Insights and Ideas

  • Decomposition is Key: The core insight is that tackling the complex lane-level map extraction problem by splitting it into non-intersection and intersection sub-tasks, and further decomposing intersection handling into validation and extraction, makes the problem tractable.
  • Leveraging Aerial Imagery for High-Detail Maps: This work demonstrates the potential of using readily available aerial imagery to create detailed lane-level maps, offering a scalable alternative to sensor-based mapping.
  • Importance of Lane Direction: Unlike road-level extraction, accurately inferring lane direction is critical for creating a routable lane graph, especially for one-way roads where explicit visual cues can be scarce.
  • Turning Lane Validation is Non-Trivial: Simply classifying potential turning lanes is insufficient. The authors' approach of using "reachable lane segmentation" to provide spatial context to the validation model significantly improves accuracy.

Key Results

  • Lane Extraction at Non-Intersections: The proposed model achieved a GEO F1-score of 0.828 and a TOPO F1-score of 0.748 on undirected lane extraction at non-intersection areas, outperforming baseline UNet models.
  • Lane Direction Inference: The lane direction inference accuracy was high, reaching 94.56% overall, though accuracy on one-way roads (around 70%) was notably lower than on two-way roads (around 98%).
  • Turning Lane Validation: The authors' full solution achieved a high F1-score of 94.1% for turning lane validation, significantly outperforming simpler classification or segmentation-only alternatives.
  • Overall Performance: The complete pipeline achieved a GEO F1-score of 0.765 and a TOPO F1-score of 0.627 on directed lane graphs. While these numbers might seem lower, the authors note that the matching threshold of 1 meter is quite strict.
  • Qualitative Results: The paper presents compelling qualitative results showing successful extraction of complex intersections, including lane diverging, and various turning lane configurations.

Limitations and Future Work

  • Visibility Dependence: The quality of the extracted lane graph is highly dependent on the visibility of road features like lane markers. Areas with unclear or occluded markings pose a significant challenge.
  • Reliance on Ground Truth: The turning lane validation model struggles with unusual turning lanes (e.g., u-turns) and cases where road signs are crucial for interpretation, suggesting a need for integrating external information.
  • Future Work: The authors suggest estimating a confidence score for extracted lane graphs based on feature visibility. This could allow for more robust integration with other mapping solutions or provide users with an indication of map reliability. They also propose using this confidence score to complement other mapping solutions.

Summary updated

BibTeX
@inproceedings{he2022lane,
  author = {Songtao He and Hari Balakrishnan},
  title = {{Lane-Level Street Map Extraction From Aerial Imagery}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2022},
}
Figure