Automatic Road Pavement Distress Recognition Using Deep Learning Networks from Unmanned Aerial Imagery

Farhad Samadzadegan, Farzaneh Dadrass Javan, Farnaz Ashtari Mahini, Mehrnaz Gholamshahi, Francesco NexDrones 2024 · OpenReview

TL;DR

This paper presents a robust deep learning approach using the YOLOv8 network for automatically identifying seven common road pavement distress types from both aerial and terrestrial imagery. By leveraging a large, diverse dataset and an optimized network architecture, the method achieves high accuracy, paving the way for more efficient and cost-effective infrastructure maintenance.

What Problem Does This Paper Address?

Maintaining the integrity of road infrastructure is critical for economic activity and public safety. However, road pavements degrade over time due to various factors like weather, heavy traffic, and natural disasters, leading to various "distresses" such as cracks, potholes, and delamination. Manually identifying and classifying these distresses is labor-intensive, time-consuming, and prone to human error. Existing automated methods often struggle with challenges like crowded backgrounds, the presence of multiple distress types in a single image, and the small size of some distresses. This paper aims to address these limitations by developing an automated system that can accurately and efficiently recognize a comprehensive set of pavement distresses from diverse image sources.

Methods

The core of this research is the application of the YOLOv8 deep learning network, a state-of-the-art single-stage object detection model. YOLOv8 was chosen for its optimized architecture, which includes multi-scale feature extraction and an anchor-free approach, contributing to improved accuracy, speed, and robustness. The authors specifically highlight the network's ability to combine high-level semantic features with low-level ones, enhancing its capacity to distinguish between different distress types and overcome challenges like small object sizes and cluttered backgrounds. The implementation involved modifications to the YOLOv5 architecture, incorporating features like CSPDarknet backbone, PAFPN neck, and a decoupled head for prediction. The Adam optimizer was used for training, and the Mish activation function was employed for its beneficial properties in neural networks.

Datasets

The study utilized a comprehensive dataset comprising 5796 aerial and terrestrial images. This dataset was curated from various sources, including public images and videos from Google and YouTube, as well as images captured by an ADTi 26MP camera and a multirotor drone. A key aspect of their data preparation was the combination of both aerial and terrestrial images to improve the model's generalizability. The dataset was divided with 70% allocated for training and 30% for testing. Data augmentation techniques, specifically the mosaic method, were employed to enrich the training data and improve the network's robustness.

Key Insights and Ideas

  • The power of combined data: Using both aerial and terrestrial images in a single dataset significantly improves the generalizability of the model, as each data type offers unique perspectives and challenges.
  • YOLOv8's adaptability: The YOLOv8 network, with its optimized architecture and anchor-free approach, proves highly effective in handling complex scenarios like crowded backgrounds, small distresses, and varying lighting conditions.
  • Addressing real-world complexity: The study deliberately included challenging image conditions (shadows, low contrast, multiple distresses) to ensure the model's practical applicability in real-world infrastructure monitoring.
  • Beyond simple detection: The goal is not just to detect distress but to accurately recognize and categorize them, which is crucial for informed maintenance decisions.

Key Results

The implemented YOLOv8 model demonstrated strong performance across seven distress types: transverse cracks, longitudinal cracks, alligator cracks, oblique cracks, potholes, repairs, and delamination. The overall evaluation metrics were:

  • Precision: 77%
  • Accuracy: 81%
  • mAP (mean Average Precision): 79%
  • F1-Score: 74%
  • Recall: 75%

The model performed particularly well in recognizing longitudinal cracks, achieving an mAP of 87%, recall of 83%, f1-score of 86%, and precision of 92%. The study also highlighted that the model's weakest performance was in the repair class (precision of 60%), likely due to its visual similarity to other distress types. Qualitative results showed successful recognition of distresses in challenging conditions, including shadows, low contrast, and crowded backgrounds.

Limitations and Future Work

The authors acknowledge that the similarity between certain distress types (e.g., potholes and delamination, repairs and cracks) can still lead to confusion, even for the model. They also note that the patch-based labeling process of YOLO might impact accuracy.

Future work suggested includes:

  • Exploring semantic segmentation-based methods and skeleton process implementation to directly extract distress shapes.
  • Expanding the dataset to include more types of distresses.
  • Moving beyond 2D analysis to determine the depth of distresses in three dimensions.

Summary updated

Abstract

Detecting and recognizing distress types on road pavement is crucial for selecting the most appropriate methods to repair, maintain, prevent further damage, and ensure the smooth functioning of daily activities. However, this task presents challenges, such as dealing with crowded backgrounds, the presence of multiple distress types in images, and their small sizes. In this study, the YOLOv8 network, a cutting-edge single-stage model, is employed to recognize seven common pavement distress types, including transverse cracks, longitudinal cracks, alligator cracks, oblique cracks, potholes, repairs, and delamination, using a dataset comprising 5796 terrestrial and unmanned aerial images. The network’s optimized architecture and multiple convolutional layers facilitate the extraction of high-level semantic features, enhancing algorithm accuracy, speed, and robustness. By combining high and low semantic features, the network achieves improved accuracy in addressing challenges and distinguishing between different distress types. The implemented Convolutional Neural Network demonstrates a recognition precision of 77%, accuracy of 81%, mAP of 79%, f1-score of 74%, and recall of 75%, underscoring the model’s effectiveness in recognizing various pavement distress forms in both aerial and terrestrial images. These results highlight the model’s satisfactory performance and its potential for effectively recognizing and categorizing pavement distress for efficient infrastructure maintenance and management.
BibTeX
@inproceedings{samadzadegan2024automatic,
  author = {Farhad Samadzadegan and Farzaneh Dadrass Javan and Farnaz Ashtari Mahini and Mehrnaz Gholamshahi and Francesco Nex},
  title = {{Automatic Road Pavement Distress Recognition Using Deep Learning Networks from Unmanned Aerial Imagery}},
  booktitle = {Drones},
  year = {2024},
  url = {https://openreview.net/forum?id=vO5x0KAzM7},
}
Figure