TL;DR
This paper introduces RNGDet, a novel end-to-end transformer-based approach for automatically detecting road network graphs from aerial images. By iteratively generating graph vertices and edges, RNGDet achieves state-of-the-art performance, particularly in accurately representing complex road intersections and maintaining topological correctness, which is crucial for autonomous driving applications.
What Problem Does This Paper Address?
Accurate road network information is vital for autonomous vehicle navigation, motion planning, and understanding urban environments. Traditionally, this information is obtained through manual annotation, which is labor-intensive and time-consuming, especially for large areas. While automated methods exist, they often struggle with topological correctness (e.g., accurately connecting road segments at intersections) or precise detection of road structures. Segmentation-based methods can achieve good pixel-level accuracy but often fail to capture the correct graph topology. Conversely, existing graph-based methods, while better at topology, are often multi-stage and difficult to optimize end-to-end, leading to accumulated errors and inefficiencies. This paper aims to bridge this gap by proposing an end-to-end trainable system that excels in both precise detection and topological accuracy.
Methods
The core of RNGDet is a transformer-based architecture inspired by DETR, adapted for iterative graph generation. The process begins by identifying candidate initial vertices. From each vertex, the system iteratively predicts adjacent vertices. This is achieved by:
- Cropping and Rasterizing: A region of interest (ROI) is cropped from the aerial image, and the current road network graph is rasterized into a historical trajectory map (H).
- Feature Extraction: A CNN backbone (e.g., ResNet) extracts features from the ROI (F₁) and from the combined historical map, segmentation maps of road segments (S), and road intersections (I) (FH). These features are concatenated into a final feature tensor (F).
- Transformer Prediction: A transformer network, using learned vertex queries, predicts potential adjacent vertices for the current vertex. These predictions include a validity probability and coordinates.
- Graph Update: Based on the predicted valid vertices, the graph is updated. If multiple valid vertices are predicted, they are added to a candidate set for future exploration. If only one is predicted, the agent moves to that vertex. If none are predicted, the system may stop or pick a new initial vertex.
- Imitation Learning: The model is trained using imitation learning (specifically, behavior cloning) to mimic an expert's actions in exploring and constructing the road network graph. A novel sampling algorithm is proposed to generate training data by exploring the ground-truth graph.
Datasets
The paper uses the RoadTracer dataset for both training and evaluation. This dataset consists of 300 high-resolution aerial images (60cm/pixel) with ground-truth road network graphs derived from OpenStreetMap. The dataset covers 40 cities and is provided in an image coordinate system.
Key Insights and Ideas
- Transformer for Iterative Graph Generation: Leveraging transformers with deep vertex queries allows for direct prediction of multiple adjacent vertices simultaneously, effectively handling complex intersections with arbitrary numbers of road segments.
- End-to-End Trainability: Unlike previous graph-based methods that often involve separate stages, RNGDet is designed for end-to-end optimization, reducing error accumulation and improving efficiency.
- Imitation Learning for Robust Exploration: Training with imitation learning, guided by a novel sampling algorithm, helps the agent learn to explore road networks effectively and robustly, even in complex scenarios.
- Importance of Segmentation Maps: The ablation studies highlight that both road segment and road intersection segmentation maps are crucial for providing strong supervision to the CNN backbone, significantly improving performance, especially for topological correctness.
Key Results
RNGDet achieves state-of-the-art performance across various metrics, including pixel-precision (P-P), pixel-recall (P-R), pixel-F1-score (P-F), intersection-precision (I-P), intersection-recall (I-R), intersection-F1-score (I-F), and average path length similarity (APLS).
- Quantitative Improvements: RNGDet with a ResNet-101 backbone shows a 2-5% improvement on almost all metrics compared to previous state-of-the-art approaches. For instance, it achieves an APLS score of 67.88 with 10 queries, outperforming VecRoad (63.76) and Sat2Graph (63.21) in qualitative demonstrations.
- Qualitative Performance: Visualizations demonstrate RNGDet's ability to accurately detect road networks, including complex intersections, with better topological correctness than segmentation-based methods.
Limitations and Future Work
- Handling Occlusions and Complex Scenarios: The paper acknowledges that RNGDet still struggles with highly occluded overpasses and severe tree occlusions.
- Future Directions: The authors suggest improving RNGDet by using more powerful backbone networks and advanced training strategies. They also plan to adapt the approach to other graph detection tasks, such as road lane line detection and road lane centerline detection.
Summary updated