SEMANTIC SEGMENTATION AND UNREGISTERED BUILDING DETECTION FROM UAV IMAGES USING A DECONVOLUTIONAL NETWORK

Seokil Ham, Yujin Oh, Wonseok Choi, I. Lee2018 · DOI

TL;DR

This paper presents a deep learning system for automatically detecting unregistered buildings from UAV imagery. By training a deconvolutional network on publicly available national geospatial data, the system can semantically segment UAV images into building probability maps, which are then compared with existing GIS data to identify discrepancies. This approach offers a more efficient and cost-effective solution for urban management tasks like illegal building monitoring.

What Problem Does This Paper Address?

Urban management, particularly the detection of illegal buildings (e.g., in green belts) and the updating of Geographic Information Systems (GIS) databases, is a crucial but often labor-intensive task. While Unmanned Aerial Vehicles (UAVs) offer a more frequent and efficient way to capture high-resolution aerial imagery compared to traditional manned aircraft, manually interpreting these images to identify unregistered structures remains time-consuming and costly. This paper aims to bridge this gap by developing an automated system that leverages deep learning to detect these unregistered buildings from UAV data.

Methods

The core of the proposed system is a deconvolutional network used for semantic segmentation. This choice is motivated by the limitations of previous methods like patch-based CNNs (which can produce boundary artifacts) and Fully Convolutional Networks (FCNs) (which struggle with objects of significantly different scales). Deconvolutional networks, with their unpooling and deconvolution layers, are better equipped to handle varying object sizes and can naturally densify segmentation maps.

The authors adapted a deconvolutional network architecture based on a mirrored version of VGG-16. The training process involves minimizing a cross-entropy loss function using the Adam optimizer.

For the detection of unregistered buildings, the process involves:

  1. Semantic Segmentation: The trained deconvolutional network generates a building probability map from a given UAV image.
  2. Comparison with GIS Data: This probability map is then compared with existing GIS building data.
  3. Post-processing: An "opening operation" (a morphological operation combining erosion and dilation) is applied to remove noise and refine the detected regions, thereby identifying potential unregistered buildings.

Datasets

  • Training and Validation: The authors utilized publicly available national geospatial data from the Korean National Geographic Information Institute (NGII). This data includes aerial ortho-images and corresponding digital maps (building and road layers). The authors acknowledge this data is "weakly labeled" due to inconsistencies in update periods and potential relief displacements in high-rise buildings.
  • Acquisition of UAV Images: The system was tested on UAV images acquired over Gangsang-myeon, Yangpyeong-gun, Gyeonggi-do, Korea. A DJI Mavic Pro drone captured 154 images covering 0.28 km² at an altitude of 120m, resulting in a Ground Sampling Distance (GSD) of 5.25cm. These images were mosaicked into an ortho-image and registered to existing GIS data.

Key Insights and Ideas

  • Publicly Available Data Sufficiency: The study demonstrates that publicly available, albeit "weakly labeled," geospatial data can be sufficiently effective for training deep learning models for semantic segmentation of aerial imagery, reducing the need for costly manual annotation.
  • Deconvolutional Networks for Scale Invariance: The choice of a deconvolutional network proved effective in handling the diverse sizes and shapes of buildings present in aerial images, overcoming limitations of earlier segmentation approaches.
  • Morphological Operations for Refinement: The simple yet effective use of an opening operation as a post-processing step is crucial for cleaning up segmentation results and accurately identifying unregistered buildings by removing noise.
  • UAVs as a Viable Platform: The research reinforces the utility of UAVs for high-resolution aerial data acquisition, enabling more detailed and frequent monitoring for urban management.

Key Results

  • Detection Accuracy: The system successfully detected 85% of unregistered buildings that were manually digitized.
  • Quantitative Evaluation: The recall-precision curve showed a breakeven point of 71% for building detection and 78% for road detection.
  • Qualitative Findings: The system performed well in normal cases but struggled with high-rise buildings exhibiting relief displacements and buildings with ambiguous appearances (e.g., those with integrated sports facilities).
  • False Positives: The system also detected extended buildings (7 instances), greenhouses (4 instances, with ambiguous results), and other objects like parking lots and farm equipment (8 instances of "wrong detection"), indicating areas for improvement.
  • Training Time: Training the deconvolutional network for 15 epochs on an Nvidia GTX 1080 Ti GPU took approximately 2 days and 11 hours.

Limitations and Future Work

  • Limitations of Training Data: The "weakly labeled" nature of the national geospatial data, particularly the relief displacements in high-rise buildings and the classification of ambiguous structures, led to segmentation errors.
  • Ambiguous Objects: The system sometimes misclassified greenhouses and other objects like parking lots as buildings, impacting performance.
  • Future Work: The authors suggest that the proposed system can be extended to monitor illegal building extensions and applied to various urban management tasks, including monitoring illegal buildings in green belt areas, detecting illegal land-use changes, and updating digital maps. Further research could focus on improving the handling of complex building structures and reducing false positives.

Summary updated

Abstract

Detecting unregistered buildings from aerial images is an important task for urban management such as inspection of illegal buildings in green belt or update of GIS database. Moreover, the data acquisition platform of photogrammetry is evolving from manned aircraft to UAVs (Unmanned Aerial Vehicles). However, it is very costly and time-consuming to detect unregistered buildings from UAV images since the interpretation of aerial images still relies on manual efforts. To overcome this problem, we propose a system which automatically detects unregistered buildings from UAV images based on deep learning methods. Specifically, we train a deconvolutional network with publicly opened geospatial data, semantically segment a given UAV image into a building probability map and compare the building map with existing GIS data. Through this procedure, we could detect unregistered buildings from UAV images automatically and efficiently. We expect that the proposed system can be applied for various urban management tasks such as monitoring illegal buildings or illegal land-use change.
BibTeX
@inproceedings{ham2018semantic,
  author = {Seokil Ham and Yujin Oh and Wonseok Choi and I. Lee},
  title = {{SEMANTIC SEGMENTATION AND UNREGISTERED BUILDING DETECTION FROM UAV IMAGES USING A DECONVOLUTIONAL NETWORK}},
  booktitle = {˜The},
  year = {2018},
  doi = {10.5194/isprs-archives-xlii-2-419-2018},
}
Figure