Semantic Segmentation of Building Roof in Dense Urban Environment with Deep Convolutional Neural Network: A Case Study Using GF2 VHR Imagery in China

Yuchu Qin, Yunchao Wu, Bin Li, Shuai Gao, Miao Liu, Yulin Zhan2019 · DOI

TL;DR

This paper introduces a novel Deep Convolutional Neural Network (DCNN) approach for accurately segmenting building roofs in dense urban environments using Chinese GF2 Very High Resolution (VHR) satellite imagery. The method achieves high accuracy, paving the way for automated, large-scale building mapping crucial for urban planning and sustainable development.

What Problem Does This Paper Address?

Accurate and up-to-date building maps are essential for urban planning, environmental management, and monitoring sustainable development goals. Traditional methods like manual surveys or image annotation are time-consuming and labor-intensive, especially in rapidly urbanizing areas with diverse building types. While Very High Resolution (VHR) satellite imagery offers a promising alternative, effectively extracting building roofs from such data, particularly in complex, dense urban landscapes, remains a challenge. Existing methods often rely on hand-crafted features or specific rules, making them difficult to generalize across different urban environments. This paper aims to bridge this gap by developing an automated, end-to-end DCNN-based approach for building roof segmentation using affordable GF2 VHR imagery.

Methods

The core of this research is a custom-designed Deep Convolutional Neural Network (DCNN) architecture for semantic segmentation. The network is based on the VGG-16 model, known for its effectiveness in vision tasks, but adapted for pixel-level prediction. Key modifications include:

  • Fully Convolutional Network (FCN) architecture: This allows the network to output a segmentation map of the same size as the input image, crucial for dense prediction tasks.
  • Deconvolutional Layers: These layers are used to upsample the feature maps, recovering the spatial resolution lost during convolutional and pooling operations, enabling per-pixel classification.
  • Batch Normalization: Added to improve computational efficiency and segmentation accuracy by normalizing feature distributions between layers.
  • Post-processing with Conditional Random Field (CRF): A fully connected CRF model is employed to refine the segmentation results. CRFs are probabilistic graphical models that leverage spatial-spectral relationships between pixels to smooth boundaries and improve the overall coherence of the segmentation.

The DCNN model performs feature extraction and per-pixel labeling, while the CRF acts as a regularization step to enhance the final output.

Datasets

The study utilizes Chinese Very High Resolution (VHR) satellite imagery from the Gaofen-2 (GF2) Panchromatic and Multispectral Sensor (PMS).

  • Imagery: GF2 PMS imagery with a spatial resolution of 0.8 m (panchromatic) and 3.2 m (multispectral).
  • Study Areas: Seven typical cities across China were selected to represent diverse urban patterns and building styles: Beijing, Shenyang, Chengdu, Guangzhou, Wuhan, Shanghai, and Urumqi.
  • Data Collection: Training and testing data were collected through manual delineation of building roofs on GF2 PMS images. These manually delineated polygons were converted into binary images, where pixels representing building roofs were labeled as 1 and others as 0.
  • Sample Size: 1460 sample images were used, with 80% for training and 20% for validation.

The paper does not introduce a new benchmark dataset but rather uses existing GF2 imagery and manually created ground truth for its study areas.

Key Insights and Ideas

  • DCNNs for Dense Urban Environments: The study demonstrates that DCNNs are highly effective for semantic segmentation of building roofs, even in complex and dense urban settings with varied building types.
  • GF2 Imagery Potential: The research highlights the significant potential of affordable GF2 VHR satellite imagery for large-scale building mapping, making advanced remote sensing applications more accessible.
  • End-to-End Approach: The proposed FCN-based DCNN offers an end-to-end solution, simplifying the workflow from raw imagery to building roof maps.
  • CRF Refinement Benefits: While the DCNN alone achieves high accuracy, the CRF post-processing further refines the segmentation, resulting in smoother boundaries and improved visual quality, though the quantitative impact on accuracy metrics was marginal in this study.
  • Data Requirements: The success of DCNNs hinges on large volumes of both positive and negative samples, which can be costly and challenging to collect comprehensively for all urban scenarios.

Key Results

The DCNN-based semantic segmentation approach yielded impressive results:

  • Overall Accuracy (OA): The DCNN model achieved an OA of 94.67%.
  • Mean Intersection Over Union (mIOU): The mIOU was 0.83.
  • CRF-Refined Results: The CRF-refined segmentation showed a slight improvement in OA to 94.69%, with the mIOU remaining at 0.83.
  • Qualitative Assessment: Visual inspection of the segmentation results across various urban environments (dense business districts, apartment complexes, single-family homes) showed that the DCNN could generate promising building roof maps with segmented buildings appearing as connected objects. The CRF refinement resulted in smoother boundaries.

Limitations and Future Work

The authors acknowledge several limitations and propose future research directions:

  • Boundary Smoothing: While CRF helps, convolutional and pooling operations can still smooth building boundaries, potentially losing fine details.
  • Data Collection Cost: The reliance on large, manually annotated datasets is costly and difficult to scale for comprehensive coverage of diverse urban environments.
  • Future Work:
    • Algorithm Improvement: Focus on fusing low-level geometrical features, employing multi-scale feature fusion with dilated convolutional filters, and exploring image transformation for building edge enhancement.
  • Alternative Data Sources: Investigate the use of SpaceNet and Volunteering Geographical Inventory (VGI) approaches for large-scale sample collection to overcome the cost and difficulty of manual annotation.
  • Operational Pipeline: Develop an automated pipeline for operational built-up area monitoring and timely building roof map updates, leveraging the advances in high-performance computing and affordable GF2 imagery.

Summary updated

BibTeX
@misc{qin2019semantic,
  author = {Yuchu Qin and Yunchao Wu and Bin Li and Shuai Gao and Miao Liu and Yulin Zhan},
  title = {{Semantic Segmentation of Building Roof in Dense Urban Environment with Deep Convolutional Neural Network: A Case Study Using GF2 VHR Imagery in China}},
  year = {2019},
  doi = {10.3390/s19051164},
}
Figure