A CAPSNETS APPROACH TO PAVEMENT CRACK DETECTION USING MOBILE LASER SCANNNING POINT CLOUDS

Weidong Zhu, Weikai Tan, Lingfei Ma, David Zhang, Jonathan Li, Michael A. Chapman2021 · DOI

Abstract

Routine pavement inspection is crucial to keep roads safe and reduce traffic accidents. However, traditional practices in pavement inspection are labour-intensive and time-consuming. Mobile laser scanning (MLS) has proven a rapid way for collecting a large number of highly dense point clouds covering roadway surfaces. Handling a huge amount of unstructured point clouds is still a very challenging task. In this paper, we propose an effective approach for pavement crack detection using MLS point clouds. Road surface points are first converted into intensity images to improve processing efficiency. Then, a Capsule Neural Network (CapsNet) is developed to classify the road points for pavement crack detection. Quantitative evaluation results showed that our method achieved the recall, precision, and F1-score of 95.3%, 81.1%, and 88.2% in the testing scene, respectively, which demonstrated the proposed CapsNet framework can accurately and robustly detect pavement cracks in complex urban road environments.

Introduction

Maintaining road infrastructure is a high-stakes, perpetual task. Small cracks, if left undetected, allow water to infiltrate lower structural layers, leading to severe failure and safety hazards. Traditionally, inspections are performed manually—a process that is slow, subjective, and inherently dangerous for personnel working in active traffic.

Mobile Laser Scanning (MLS) offers a solution by rapidly collecting millions of 3D data points (point clouds) from moving vehicles. However, processing this "unstructured" data is computationally intensive. Standard computer vision techniques often struggle with the irregular shapes of cracks or the sheer volume of 3D data. This research proposes an efficient pipeline that transforms raw laser data into actionable maintenance maps using a deep learning architecture known as Capsule Neural Networks (CapsNets).

Methodology

The proposed workflow combines traditional geoprocessing with advanced deep learning to bridge the gap between raw point clouds and crack detection.

Proposed methodology flowchart for pavement crack detection.

1. Road Segmentation and Transformation

To optimize processing, the system first isolates the road surface from environmental noise (e.g., trees, buildings) using a curb-line-based method. Instead of processing raw 3D points, the points are projected onto a 2D plane to create georeferenced intensity images. Using Inverse Distance Weighted (IDW) interpolation, pixel values are determined based on the laser's return intensity. This is effective because cracks reflect laser light differently than smooth pavement, making them visible even without color data.

2. Capsule Neural Networks (CapsNet)

Unlike standard Convolutional Neural Networks (CNNs), which can lose track of the spatial relationship between features, CapsNets use "capsules"—groups of neurons that represent the properties and orientation of an object. This makes the model more robust at recognizing the irregular, branching patterns of pavement cracks. The model employs a "routing by agreement" mechanism, where lower-level capsules "vote" for the presence of a crack in higher layers, improving classification accuracy over multiple iterations.

Experimental Setup and Datasets

The study utilized a custom Pavement Crack (PC) dataset derived from a survey using a RIEGL VMX-450 MLS system. The data was collected from a 105-meter stretch of a two-lane cement-paved road, totaling approximately 8.4 million points.

For model training, the data was converted into intensity images and sliced into 28x28 pixel patches. The dataset was split into 1,000 patches for training, 316 for validation, and 176 for testing (a 67/21/12% split).

Results and Discussion

The CapsNet approach demonstrated strong performance in detecting cracks within complex urban environments. The model achieved a recall of 95.3%, indicating it is highly effective at identifying nearly all existing cracks. While precision was slightly lower at 81.1% due to some false positives, the overall F1-score of 88.2% reflects a balanced and effective detection capability.

Visual results of detected pavement cracks highlighted on the road surface intensity image.

Key findings from the study include:

  • Intensity Superiority: Laser reflection intensity is a more reliable indicator of cracks than RGB images, as it is less affected by shadows or ambient lighting changes.
  • Spatial Hierarchy: CapsNets successfully addressed the "Picasso problem" common in CNNs, ensuring that the components of a crack fit together spatially rather than just appearing as isolated features.
  • Iteration Impact: Increasing the number of routing iterations from 1 to 3 significantly boosted the F1-score from 81.4% to 88.2%, though at the cost of increased computational time.

Conclusion

This research demonstrates that combining MLS data with Capsule Neural Networks provides a robust framework for automated pavement inspection. While the model showed high accuracy, future work will focus on mitigating overfitting risks and optimizing the computational cost of the dynamic routing mechanism. The ultimate goal is to move toward a real-time "drive-and-detect" system that can process raw data formats directly, further enhancing the efficiency of road maintenance operations.

Summary updated

Abstract

Routine pavement inspection is crucial to keep roads safe and reduce traffic accidents. However, traditional practices in pavement inspection are labour-intensive and time-consuming. Mobile laser scanning (MLS) has proven a rapid way for collecting a large number of highly dense point clouds covering roadway surfaces. Handling a huge amount of unstructured point clouds is still a very challenging task. In this paper, we propose an effective approach for pavement crack detection using MLS point clouds. Road surface points are first converted into intensity images to improve processing efficiency. Then, a Capsule Neural Network (CapsNet) is developed to classify the road points for pavement crack detection. Quantitative evaluation results showed that our method achieved the recall, precision, and F1-score of 95.3%, 81.1%, and 88.2% in the testing scene, respectively, which demonstrated the proposed CapsNet framework can accurately and robustly detect pavement cracks in complex urban road environments.
BibTeX
@inproceedings{zhu2021capsnets,
  author = {Weidong Zhu and Weikai Tan and Lingfei Ma and David Zhang and Jonathan Li and Michael A. Chapman},
  title = {{A CAPSNETS APPROACH TO PAVEMENT CRACK DETECTION USING MOBILE LASER SCANNNING POINT CLOUDS}},
  booktitle = {˜The},
  year = {2021},
  doi = {10.5194/isprs-archives-xliii-b1-2021-39-2021},
}
Figure