Abstract
Automatic map extraction is of great importance to urban computing and location-based services. Aerial image and GPS trajectory data refer to two different data sources that could be leveraged to generate the map, although they carry different types of information. Most previous works on data fusion between aerial images and data from auxiliary sensors do not fully utilize the information of both modalities and hence suffer from the issue of information loss. We propose a deep convolutional neural network called DeepDualMapper which fuses the aerial image and trajectory data in a more seamless manner to extract the digital map. We design a gated fusion module to explicitly control the information flows from both modalities in a complementary-aware manner. Moreover, we propose a novel densely supervised refinement decoder to generate the prediction in a coarse-to-fine way. Our comprehensive experiments demonstrate that DeepDualMapper can fuse the information of images and trajectories much more effectively than existing approaches, and is able to generate maps with higher accuracy.
Introduction
Creating accurate, up-to-date city maps is essential for autonomous driving and urban planning. Researchers typically rely on two primary sources: aerial imagery (satellites/drones) and GPS trajectories (location pings from vehicles or mobile devices). However, both modalities possess inherent limitations. Aerial images are frequently obstructed by tree canopies, tall buildings, or shadows. Conversely, GPS data is often sparse, particularly for minor or less-traveled roads, making it difficult to reconstruct a complete network.
Previous attempts to combine these sources often relied on simple concatenation, failing to account for the varying reliability of each source across different urban contexts. DeepDualMapper addresses this by intelligently selecting and fusing information based on local data quality.
Methodology
DeepDualMapper is built upon a U-Net architecture and introduces two primary innovations to improve data fusion and map quality:
- Gated Fusion Module (GFM): This module acts as the system's decision-maker. Rather than treating both sources equally, the GFM calculates a learnable weight (or "gate") for each pixel. If a road is obscured by trees in an aerial image, the gate shifts its reliance to GPS data. If GPS data is missing for a new road visible in imagery, the model prioritizes the visual information.
- Densely Supervised Refinement Decoder (DSRD): To combat the noise common in map extraction—such as fragmented roads or isolated artifacts—this decoder employs a coarse-to-fine strategy. It utilizes residual refinement across five different scales to ensure road connectivity and smoothness.
- Space-Consistent Supervision: A shared prediction module is used to ensure that image and trajectory features are mapped into a uniform mathematical space, facilitating effective communication between the two modalities.
Experimental Setup
The model was evaluated using three large-scale, real-world datasets with ground truth derived from OpenStreetMap (OSM):
- Porto, Portugal: High-quality satellite imagery paired with taxi trajectories.
- Shanghai, China: A dense urban environment featuring complex road networks.
- Singapore: A city characterized by high GPS noise and significant tree occlusion.
Results and Discussion
DeepDualMapper consistently outperformed state-of-the-art baselines, such as TriSeg and 1D Decoder, across all test cities. In Porto, the model achieved an Intersection over Union (IoU) of 0.717, a significant improvement over the best baseline's 0.682. The model is also computationally efficient, processing a 224x224 patch in approximately 7.6ms on a standard consumer GPU.
Robustness to Information Loss
The researchers tested the model's resilience by intentionally removing 25% of the data from both sources. DeepDualMapper maintained an IoU of 0.588 in Porto, while competing fusion methods saw their performance drop below 0.50. This demonstrates the effectiveness of the gated mechanism in handling incomplete data.
Ground Truth Correction
A notable finding was the model's ability to identify roads missing from the "official" OpenStreetMap ground truth. Manual inspection confirmed that these predicted roads actually existed, suggesting that the model can serve as a tool for updating and correcting existing map databases.
Conclusion
DeepDualMapper provides a robust framework for automatic map extraction by effectively managing the trade-offs between aerial imagery and GPS trajectories. While the current model focuses on trajectory density, future work could incorporate velocity and heading data to further distinguish road types. Additionally, refining the model to predict varying road widths could enhance its real-world utility for urban navigation systems.
Summary updated