Vision and Deep Learning-Based Algorithms to Detect and Quantify Cracks on Concrete Surfaces from UAV Videos

Sutanu Bhowmick, Satish Nagarajaiah, Ashok Veeraraghavan2020 · DOI

TL;DR

This paper introduces a novel framework using Unmanned Aerial Vehicle (UAV) video and deep learning (specifically U-Net) to automatically detect and quantify cracks on concrete structures. This approach offers a more efficient, accurate, and less error-prone alternative to manual inspections, crucial for assessing the structural integrity of critical infrastructure.

What Problem Does This Paper Address?

Assessing the structural integrity of civil infrastructure like bridges and dams is vital, especially after natural disasters. Traditional methods rely on manual inspections by engineers, which are time-consuming, prone to human error, and often struggle to access all parts of large structures. While sensors can provide global structural behavior, they may not pinpoint local damage effectively. This paper addresses the need for an automated, efficient, and precise method for detecting and quantifying cracks on concrete surfaces, particularly from challenging vantage points, using readily available video data.

Methods

The core of the proposed methodology lies in a two-stage process:

  1. Deep Segmentation with U-Net: The authors employ a U-Net deep neural network architecture for image segmentation. U-Net is chosen for its effectiveness in segmenting biomedical images with limited annotated data, making it suitable for crack detection where manual pixel-level annotation is labor-intensive. The network is trained on RGB images of concrete surfaces and their corresponding pixel-level ground truth masks to classify each pixel as either crack or non-crack. Data augmentation techniques (rotations, flips) are used to increase the dataset's variability and improve model robustness.
  2. Morphological Operations for Crack Characterization: Following the U-Net segmentation, morphological operations are applied to the resulting binary crack masks. This process extracts detailed geometric properties of individual cracks, including:
  3. Length: Determined by tracing the skeleton of the crack.
  4. Width: Calculated by measuring the distance from the crack's outline to its skeleton.
  5. Area: Derived from the segmented crack pixels.
  6. Dominant Orientation: Identified using the RANSAC (Random Sample Consensus) algorithm, which iteratively fits lines to crack skeletons to determine their primary direction, crucial for inferring crack type (e.g., flexure vs. shear).

Datasets

The paper utilizes a combination of publicly available datasets and custom data for training and validation:

  • Partial Crack Dataset [25]: Consists of 136 RGB images of concrete cracks with pixel-level annotated ground truth.
  • Concrete Crack Images [26]: 69 RGB images of cracked concrete surfaces.

These datasets were pre-processed, including cropping, resizing to 256x256 pixels, and extensive data augmentation (rotations, flips). A total of 1700 augmented images were used for training, and 346 for validation. The experimental validation used video data from a four-point bending test on an 8-foot concrete beam, captured by both a UAV-mounted camera and a ground-based iPhone camera. Importantly, the images from this experimental validation were not part of the training or validation sets, ensuring an unbiased assessment of the model's robustness.

Key Insights and Ideas

  • U-Net's Efficacy with Limited Data: The study demonstrates that U-Net can achieve high crack segmentation accuracy with a relatively small number of pixel-annotated images (346 for training/validation), significantly reducing the annotation burden compared to other deep learning models.
  • Morphological Operations for Richer Information: Moving beyond simple crack detection, the paper highlights how morphological operations can extract detailed geometric features (length, width, area, orientation) that are essential for a comprehensive structural assessment.
  • RANSAC for Robust Crack Orientation: The use of RANSAC for determining crack orientation is a key innovation, allowing for the identification of dominant directions even in complex cracks that might be a combination of flexure and shear, providing critical insights into crack behavior.
  • UAVs as a Practical Inspection Tool: The experimental validation using UAV video underscores the practical applicability of the framework for inspecting hard-to-reach areas and capturing dynamic crack evolution in real-time.
  • Unbiased Validation: Using experimental data entirely separate from the training set is a strong methodological choice that validates the model's generalization capabilities.

Key Results

  • The U-Net model achieved a high IoU (Intersection-Over-Union) score of 0.6366 for crack mask segmentation, with optimal hyperparameters identified as a batch size of 10 and 30 epochs.
  • The framework successfully detected and quantified multiple cracks on a concrete beam during a bending test until failure.
  • The geometric measurements (length, width, area) of cracks were tracked over time, showing discrete jumps corresponding to the appearance of new cracks.
  • The analysis of crack orientation successfully differentiated between initial flexural cracks (around π/2 radians) and later shear cracks (around π/4 radians), validating the method's ability to infer crack nature.
  • The study demonstrated that geometric measurements from UAV video, while subject to some error due to motion and perspective changes, were reasonably consistent with ground-based measurements, especially when the UAV maintained a stable position.

Limitations and Future Work

  • Segmentation of Non-Concrete Surfaces: The paper acknowledges that segmenting structural surfaces from other objects and background elements in the video frames is a separate challenge that is beyond the scope of this study and is slated for future work.
  • Geometric Measurement Accuracy: While the framework provides geometric measurements, the accuracy of these measurements from UAV video can be affected by factors like camera motion, translation, rotation, and perspective distortion, especially in the later stages of the experiment. The authors note that these measurements are converted to physical units using a scaling factor derived from known dimensions, but this approximation is valid only when the object plane is parallel to the image plane.
  • Future Work: The authors propose extending the methodology to include LiDAR integration for 3D mapping of structures and automating the segmentation of concrete surfaces from acquired video. They also aim to use the extracted crack orientation and magnitude data for integrity assessment before failure.

Summary updated

BibTeX
@misc{bhowmick2020vision,
  author = {Sutanu Bhowmick and Satish Nagarajaiah and Ashok Veeraraghavan},
  title = {{Vision and Deep Learning-Based Algorithms to Detect and Quantify Cracks on Concrete Surfaces from UAV Videos}},
  year = {2020},
  doi = {10.3390/s20216299},
}
Figure