UAV-BASED STRUCTURAL DAMAGE MAPPING – RESULTS FROM 6 YEARS OFRESEARCH IN TWO EUROPEAN PROJECTS

Norman Kerle, Francesco Nex, Diogo Duarte, Anand Vetrivel

Abstract

Structural disaster damage detection and characterisation is one of the oldest remote sensing challenges. The proliferation of UAVs has opened new opportunities for damage mapping due to high spatial resolution and platform flexibility. This paper synthesizes six years of research from the RECONASS and INACHUS projects, focusing on advanced image analysis driven by computer vision, photogrammetry, and machine learning. Initially focusing on supervised texture features and basic classifiers like SVM and Random Forest, the work transitioned to deep learning. The coupling of image-derived features and 3D point cloud information in Convolutional Neural Networks (CNNs) proved successful in detecting subtle damage features like cracks and spalling. We further developed a processing pipeline and mobile app to facilitate near-real-time damage mapping, evaluated by first responders and civil protection end users.

Introduction

When disasters such as earthquakes or explosions occur, rapid assessment of building stability is critical for Urban Search and Rescue (USAR) teams. Traditional satellite-based mapping often suffers from "nadir bias," primarily capturing roofs while missing critical facade indicators like cracks, spalling, or structural tilts. This research addresses the need for a high-resolution, multi-perspective, and automated system that can provide detailed building-level damage data in near-real time to support emergency response.

Methodology

The research reflects a decade-long evolution in remote sensing methodology, moving from manual feature engineering to automated deep learning pipelines:

  1. Texture & Geometry: Early work utilized Object-Based Image Analysis (OBIA) and hand-crafted features (e.g., Gabor wavelets) fed into traditional machine learning classifiers like Support Vector Machines (SVM).
  2. 3D Point Cloud Integration: Using Structure-from-Motion (SfM), the team generated 3D models from UAV imagery to detect geometric anomalies, such as wall deformations or missing structural segments.
  3. Deep Learning & Fusion: The current phase employs Convolutional Neural Networks (CNNs) using multi-kernel learning. This approach fuses radiometric (pixel) data with geometric (3D point cloud) features, allowing the AI to distinguish between shadows and genuine structural holes.
  4. Real-Time Pipeline: A workflow was developed to stream images from consumer drones to a local workstation, progressively building a 3D model and running damage detection in parallel.
Overview of the UAV-based damage mapping pipeline: (a) flight path reconstruction, (b) facade damage segmentation, (c) 3D point cloud generation, and (d) geometric anomaly detection.

Key Results and Insights

The transition to deep learning significantly improved detection capabilities. CNN-based facade classification reached approximately 80% accuracy, even when dealing with complex architectural features and shadows. User validation with 25 international stakeholders yielded a score of 4.5 out of 5, highlighting the practical utility of "one-button" automated apps over complex, expensive LiDAR systems.

Detailed view of automated facade damage segmentation, identifying cracks (magenta) and window openings (cyan) to assist in structural health assessment.

A critical insight is that "the facade is king"—facade data is far more descriptive of structural health than roof data, despite the challenges of occlusions. Furthermore, the research demonstrated that models trained on one disaster (e.g., Haiti) could be transferred to new scenes with minimal retraining, a prerequisite for immediate emergency response.

Conclusion and Future Work

While significant progress has been made in outdoor mapping, challenges remain in indoor, GPS-denied environments. Future research will focus on autonomous decision-making for drones to investigate detected damage more closely and bridging the "semantic gap" to understand the structural implications of detected features—distinguishing cosmetic issues from imminent total collapse.

Summary updated

Abstract

Structural disaster damage detection and characterisation is one of the oldest remote sensing challenges, and the utility of virtually every type of active and passive sensor deployed on various air- and spaceborne platforms has been assessed. The proliferation and growing sophistication of UAV in recent years has opened up many new opportunities for damage mapping, due to the high spatial resolution, the resulting stereo images and derivatives, and the flexibility of the platform. We have addressed the problem in the context of two European research projects, RECONASS and INACHUS. In this paper we synthesize and evaluate the progress of 6 years of research focused on advanced image analysis that was driven by progress in computer vision, photogrammetry and machine learning, but also by constraints imposed by the needs of first responder and other civil protection end users. The projects focused on damage to individual buildings caused by seismic activity but also explosions, and our work centred on the processing of 3D point cloud information acquired from stereo imagery. Initially focusing on the development of both supervised and unsupervised damage detection methods built on advanced texture features and basic classifiers such as Support Vector Machine and Random Forest, the work moved on to the use of deep learning. In particular the coupling of image-derived features and 3D point cloud information in a Convolutional Neural Network (CNN) proved successful in detecting also subtle damage features. In addition to the detection of standard rubble and debris, CNN-based methods were developed to detect typical façade damage indicators, such as cracks and spalling, including with a focus on multi-temporal and multi-scale feature fusion. We further developed a processing pipeline and mobile app to facilitate near-real time damage mapping. The solutions were tested in a number of pilot experiments and evaluated by a variety of stakeholders.
BibTeX
@inproceedings{kerle2019uav,
  author = {Norman Kerle and Francesco Nex and Diogo Duarte and Anand Vetrivel},
  title = {{UAV-BASED STRUCTURAL DAMAGE MAPPING – RESULTS FROM 6 YEARS OFRESEARCH IN TWO EUROPEAN PROJECTS}},
  booktitle = {˜The},
  year = {2019},
  doi = {10.5194/isprs-archives-xlii-3-w8-187-2019},
}
Figure