AI powered road network prediction with fused low-resolution satellite imagery and GPS trajectory

Necip Enes Gengeç, Ergin Tari, Ulas Bagci

TL;DR

This paper demonstrates that combining free, low-resolution satellite imagery (Sentinel-2) with GPS trajectory data can produce high-quality road network maps that rival those made with expensive high-resolution data. By testing various deep learning architectures and fusion strategies, the authors found that a "Late Fusion" approach using a ResUnet model achieves state-of-the-art results for low-resolution road extraction.

What Problem Does This Paper Address?

Accurate road network data is the backbone of modern navigation, urban planning, and disaster response. Traditionally, these maps are created via manual digitization or expensive high-resolution satellite imagery. While free satellite data like Sentinel-2 exists, its 10m/pixel resolution is often too "blurry" for precise road delineation, especially in dense urban environments.

The authors address this "resolution gap" by asking: Can we use multi-modal data fusion—specifically combining low-resolution imagery with GPS trajectories—to create accurate road maps without the high cost of premium satellite data? While GPS data has been used with high-res imagery before, its synergy with low-res, freely available data remained largely unexplored.

Methods

The researchers treated road detection as a semantic segmentation task, testing three popular U-shaped architectures: U-Net, ResUnet (which adds residual units to prevent overfitting), and D-Linknet (which uses dilated convolutions to capture more context).

The core of their experimentation lies in Fusion Strategies:

  • Early Fusion: Concatenating satellite and GPS data at the input level.
  • Late Fusion Type-1: Applying the model only to the satellite stream and then merging it with the GPS data at the end.
  • Late Fusion Type-2: Running separate model streams for both satellite and GPS data, then merging their features before the final output.

They also experimented with different "fusion operators" (Concatenate, Average, Maximum, and Multiply) and loss functions (MSE, Binary Cross-Entropy, and Focal Loss) to find the optimal configuration for handling class imbalance (since roads occupy only a small fraction of any given image).

Datasets

The study utilized data from two distinct geographic domains to test the generalizability of their models:

  1. Istanbul, Turkey: A massive dataset containing ~360 million GPS points from various vehicles.
  2. Montreal, Canada: A dataset of ~40 million GPS points from passenger cars.
  3. Sentinel-2 Satellite Imagery: 10m resolution multi-spectral data (RGB + Infrared), upscaled to 2.5m for processing.
  4. OpenStreetMap (OSM): Used as the "ground truth" labels, with road widths adjusted based on their classification (e.g., motorways vs. footways).

Key Insights and Ideas

  • GPS as a "Structural Skeleton": GPS trajectories provide a precise skeleton of human movement that compensates for the lack of visual detail in low-resolution imagery. This allows the model to "see" roads that are obscured by shadows or trees in the satellite view.
  • The Complexity Gap: The authors performed a complexity analysis using entropy and homogeneity. They found that Istanbul’s urban fabric is significantly more complex than Montreal’s. Consequently, models trained on one city didn't always generalize perfectly to the other, highlighting that "one size fits all" remains a challenge in global road mapping.
  • Boundary Matters: The authors advocate for using Boundary-IoU alongside standard IoU. Standard IoU can be misleadingly high if a model predicts a "blob" over a road; Boundary-IoU specifically checks if the edges of the predicted road align with the real ones, which is critical for actual navigation.

Key Results

  • Top Performer: The ResUnet model using Late Fusion Type-2 (with concatenation and MSE loss) achieved the best performance.
  • Quantitative Gains: This configuration reached a mean Intersection over Union (mIoU) of 0.784, significantly outperforming previous benchmarks that relied on Sentinel-2 data alone (which hovered around 0.68–0.70).
  • Fusion Wins: In almost all cases, adding GPS data improved the mIoU by a significant margin compared to satellite-only models.
  • Late vs. Early: Late fusion (merging features deep in the network) generally outperformed early fusion, suggesting that the model benefits from learning the unique features of imagery and trajectories separately before combining them.

Limitations and Future Work

  • Generalization: While the models performed well, there was a noticeable drop in accuracy (~0.2 mIoU) when a model trained on Montreal was tested on Istanbul, and vice versa. This suggests that local urban characteristics (building density, road materials, vegetation) still play a massive role.
  • Computational Cost: Late Fusion Type-2 requires running two parallel networks, which increases the memory and processing power needed during training.
  • Future Directions: The authors suggest exploring Transformer-based architectures (like SegFormer or RoadFormer) which have shown promise in other vision tasks, as well as incorporating more diverse loss functions and regularization strategies to further improve cross-city generalization.

Summary updated

Abstract

This study presents an innovative approach for automatic road detection with deep learning, employing fusion strategies to utilize both lower-resolution satellite imagery and GPS trajectory data, a concept never explored before. We rigorously investigate both early and late fusion strategies and assess deep learning-based road detection performance using different fusion settings. Our extensive ablation studies evaluate the efficacy of our framework under diverse model architectures, loss functions, and geographic domains (Istanbul and Montreal). For an unbiased and complete evaluation of road detection results, we use both region-based and boundary-based evaluation metrics for road segmentation. The outcomes reveal that the ResUnet model outperforms U-Net and D-Linknet in road extraction tasks, achieving superior results over the benchmark study using low-resolution Sentinel-2 data. This research not only contributes to the field of automatic road detection but also offers novel insights into the utilization of data fusion methods in diverse applications.
BibTeX
@inproceedings{gengec2024ai,
  author = {Necip Enes Gengeç and Ergin Tari and Ulas Bagci},
  title = {{AI powered road network prediction with fused low-resolution satellite imagery and GPS trajectory}},
  booktitle = {Earth Sci. Informatics},
  year = {2024},
  url = {https://openreview.net/forum?id=IOkxeHADz8},
}

Loading related content...

Figure