NL-LinkNet: Toward Lighter but More Accurate Road Extraction with Non-Local Operations

Yooseung Wang, Junghoon Seo, Taegyun JeonarXiv 2019 · arXiv

Abstract

Road extraction from high-resolution satellite imagery is a critical task for map updating and navigation. However, traditional Convolutional Neural Networks (CNNs) often struggle with occlusions and long-range dependencies due to limited receptive fields. This paper introduces NL-LinkNet, a lightweight architecture that integrates non-local operations to capture global spatial relationships. By strategically placing Non-Local Blocks (NLB) within a LinkNet framework, the proposed model achieves state-of-the-art performance on the DeepGlobe 2018 Road Extraction dataset. Notably, NL-LinkNet utilizes 43% fewer parameters than the previous leading model, D-LinkNet, while delivering superior road continuity and faster convergence.

Introduction

The extraction of road networks from satellite imagery faces significant challenges, primarily due to the thin, winding nature of roads and frequent occlusions from trees, shadows, and buildings. Standard CNNs rely on local receptive fields, which often fail to maintain connectivity when a road segment is obscured. While techniques like dilated convolutions expand the receptive field, they still lack the global context necessary to "connect the dots" across large spatial gaps. The authors propose a solution that provides a global perspective without the computational overhead of large ensembles or complex post-processing.

Methods

NL-LinkNet builds upon the LinkNet architecture with a ResNet34 backbone. The core innovation is the integration of Non-Local Blocks (NLB). Unlike standard convolutions that operate on a local neighborhood, a non-local operation computes the response at a position as a weighted sum of features from all positions in the input map. This provides an effectively infinite receptive field, allowing every pixel to "attend" to every other pixel.

To maintain efficiency, the authors employed $1 \times 1$ convolutions to reduce channel dimensions within the NLBs. Through empirical testing, the blocks were placed after the third and fourth stages of the encoder, where feature maps are sufficiently downsampled to manage the quadratic memory cost of non-local operations. The Embedded Gaussian function was selected as the optimal pairwise relationship function.

NL-LinkNet Architecture showing the placement of Non-Local Blocks (NLB) within the encoder stages.

Results and Discussion

The model was evaluated on the DeepGlobe 2018 Road Extraction Challenge dataset, consisting of 0.5m resolution satellite images. NL-LinkNet achieved a mean Intersection-over-Union (mIOU) of 65.00%, outperforming the previous state-of-the-art D-LinkNet (64.12%) and other benchmarks like U-Net and LinkNet.

Qualitatively, NL-LinkNet demonstrates a superior ability to preserve road continuity. In complex urban and rural scenes where traditional models produce fragmented outputs, NL-LinkNet successfully infers the path of the road by leveraging global context. If a road is blocked by a tree in one spot, the model uses the context of the road's trajectory elsewhere to infer its presence.

Qualitative comparison of road extraction results across different models. NL-LinkNet (Ours) shows significantly better continuity in challenging areas.

Beyond accuracy, the model is highly resource-efficient, requiring 43% fewer parameters than D-LinkNet. It also exhibits faster convergence during training, reaching peak performance in fewer epochs compared to its competitors. This efficiency paradox—achieving higher accuracy with fewer parameters—is attributed to the more "intelligent" spatial operation of the non-local blocks rather than simply stacking more layers.

Conclusion

NL-LinkNet proves that integrating global context via non-local operations is a more effective strategy for road extraction than simply increasing model depth or complexity. By capturing long-range dependencies, the model achieves higher accuracy with significantly fewer parameters. Future research will focus on optimizing the memory footprint of non-local operations to enable their use at higher feature map resolutions and exploring multi-spectral data integration.

Summary updated

Abstract

Road extraction from very high resolution satellite (VHR) images is one of the most important topics in the field of remote sensing. In this paper, we propose an efficient Non-Local LinkNet with non-local blocks that can grasp relations between global features. This enables each spatial feature point to refer to all other contextual information and results in more accurate road segmentation. In detail, our single model without any post-processing like CRF refinement, performed better than any other published state-of-the-art ensemble model in the official DeepGlobe Challenge. Moreover, our NL-LinkNet beat the D-LinkNet, the winner of the DeepGlobe challenge, with 43 \% less parameters, less giga floating-point operations per seconds (GFLOPs) and shorter training convergence time. We also present empirical analyses on the proper usages of non-local blocks for the baseline model.
BibTeX
@misc{wang2019nl,
  author = {Yooseung Wang and Junghoon Seo and Taegyun Jeon},
  title = {{NL-LinkNet: Toward Lighter but More Accurate Road Extraction with Non-Local Operations}},
  year = {2019},
  eprint = {1908.08223},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1908.08223},
}
Figure