TL;DR
This paper introduces csBoundary, a novel system for automatically detecting road boundaries in aerial images to create high-definition (HD) maps. It tackles the challenge of city-scale annotation by proposing a graph-generation approach that leverages transformers for efficient and accurate road boundary graph prediction, outperforming existing methods.
What Problem Does This Paper Address?
High-Definition (HD) maps are crucial for autonomous driving, providing detailed information about the road environment. Road boundaries, which delineate drivable areas from off-road regions, are a fundamental component of these maps. However, manually annotating these boundaries at a city scale is an extremely labor-intensive and time-consuming process. Existing automated methods, such as semantic segmentation or iterative graph growing, suffer from limitations like topological inaccuracies, inefficiency, and error accumulation. This paper aims to bridge this gap by developing a system that can automatically generate city-scale road boundary graphs from aerial imagery, enabling more efficient and accurate HD map creation.
Methods
csBoundary employs a graph-generation approach, directly inferring a graph of road boundaries from aerial image patches. The core of the method involves two main stages:
- Patch-level Graph Prediction:
- Keypoint and Segmentation Map Prediction: A Feature Pyramid Network (FPN) is used to predict a keypoint map (identifying potential road boundary vertices) and a road boundary segmentation map.
- Vertex Extraction: Based on the keypoint map, graph vertices are extracted. This involves skeletonizing the keypoint map and identifying local peaks. Special handling is given to "gathered keypoints" at rounded corners, where only endpoints of curved skeletons are used as vertices.
- Adjacency Matrix Prediction (AfANet): A novel network called Attention for Adjacency Network (AfANet) is introduced. AfANet uses a transformer-inspired self-attention mechanism to predict the adjacency matrix between the extracted vertices. It takes vertex embeddings (a combination of local ROI features, global image features, and vertex coordinates) as input and outputs the adjacency matrix, defining the edges of the road boundary graph for the patch.
- City-Scale Graph Stitching:
- Image Patch Expansion: To facilitate stitching, input image patches are expanded to create overlapping regions between adjacent patches.
- Graph Stitching: The predicted graphs from individual patches are then stitched together using the overlapping regions. This process ensures continuity and creates a unified city-scale road boundary graph.
Datasets
The paper utilizes the topo-boundary dataset introduced in their previous work [3]. This dataset consists of 2,049 high-resolution aerial image tiles (5000x5000 pixels) covering five boroughs of New York City. For computational efficiency, each tile is split into 25 smaller image patches (1000x1000 pixels). The Staten Island borough is reserved for testing, while the other boroughs are used for training and validation.
Key Insights and Ideas
- Transformer for Graph Prediction: The paper pioneers the use of transformers (specifically, the self-attention mechanism) for predicting the adjacency matrix of road boundary graphs. This is a significant departure from previous methods that relied on iterative processes or less efficient sequential models.
- Graph Generation without Heuristics: csBoundary directly generates the graph, avoiding the need for complex post-processing or iterative graph growing, which often leads to topological errors or inefficiencies.
- Unified Vertex Embedding: The proposed vertex embedding, combining local visual features, global image context, and precise coordinates, effectively captures rich information for accurate adjacency prediction.
- Handling Rounded Corners: The method's specific approach to extracting vertices at rounded corners, by using skeleton endpoints, is a practical solution for representing these common road features.
- Patch Expansion for Stitching: The strategy of expanding image patches to create overlapping regions is a crucial step for seamless city-scale graph stitching, addressing a key challenge in large-area mapping.
Key Results
- Superior Performance: csBoundary consistently outperforms baseline methods (OrientationRefine, Enhanced-iCurb, and Sat2Graph) across various metrics, including pixel-level (Precision, Recall, F1-score) and topology-level (APLS, TLTS) evaluations.
- City-Scale Advantage: While patch-scale performance is strong, the city-scale results demonstrate the effectiveness of the graph stitching mechanism, particularly in topology-level metrics.
- Efficiency: The proposed AfANet, leveraging transformer's parallelization capabilities, significantly reduces inference time compared to iterative methods, making city-scale annotation more feasible. Training time is also competitive.
- Qualitative Results: Visualizations show that csBoundary produces more complete and topologically correct road boundary graphs compared to other methods.
Limitations and Future Work
- Sat2Graph Adaptation: The paper notes that the original Sat2Graph method was not directly suitable for their task due to an isomorphic encoding issue. They adapted it by using their own vertex extraction, but it still underperformed. This suggests that the specific encoding scheme of Sat2Graph might be sensitive to the nature of road boundaries.
- Future Work: The authors plan to adapt AfANet to other line-shaped object detection tasks to demonstrate its generalization ability. They also aim to further explore the potential of their proposed method.
Summary updated