Detection of Road Crack Images Based on Multistage Feature Fusion and a Texture Awareness Method

Maozu Guo, Wenbo Tian, Yang Li, Dong SuiSensors 2024 · DOI

TL;DR

This paper introduces FetNet, a novel deep learning model for automatically detecting and segmenting road cracks from images. By combining a Swin Transformer backbone with a specialized texture unit and attention mechanisms, FetNet achieves state-of-the-art performance in accurately identifying cracks, even in challenging conditions with complex textures and noise.

What Problem Does This Paper Address?

Maintaining the structural integrity of roads is crucial for transportation safety and efficiency. Road cracks are a primary indicator of structural damage, but their detection and precise segmentation have historically been challenging. Traditional manual inspection methods are subjective, time-consuming, and labor-intensive. While image processing techniques have improved, they often struggle with complex noise, low contrast, and the intricate textures of cracks, leading to limitations in accuracy and efficiency. Existing deep learning methods, particularly those based on Convolutional Neural Networks (CNNs), face difficulties in capturing long-range dependencies and fine-grained crack details, especially in high-resolution images with diverse crack characteristics. This paper aims to bridge this gap by developing a robust and accurate method for pixel-level road crack segmentation that can handle these complexities.

Methods

The proposed method, named FetNet, leverages a Swin Transformer (Swin-T) as its backbone for extracting multilevel semantic features. Key novel components include:

  • Texture Unit: This module uses stacked convolutional layers to specifically capture the texture and edge information of cracks, which are crucial for distinguishing them from background noise.
  • Refinement Attention Module (RAM): FetNet employs RAM to effectively fuse features from different stages of the Swin-T backbone. It uses a Binary Attention Module (BAM) with Linear Attention (LA) and Attachment Linear Attention (ALA) to enhance spatial relationships and global context, addressing limitations of standard attention mechanisms in CNNs.
  • Panoramic Feature Module (PFM): This module further integrates the enhanced semantic features from RAM with the texture features extracted by the Texture Unit, creating a comprehensive feature representation.
  • Cascade Pyramid Architecture (CPA): CPA is used to enhance multiscale receptive fields and fuse features across different stages, improving the model's ability to recognize cracks at various scales.
  • Fully Connected Conditional Random Field (FC-CRF): Finally, an FC-CRF is applied to refine the segmentation results, smoothing boundaries and eliminating misclassifications.

Datasets

FetNet was evaluated on four publicly available road crack datasets:

  • DeepCrack: Contains 537 high-quality crack images with diverse scenes, scales, and surfaces (concrete, brick, stone).
  • GAPs384: Features 1969 asphalt crack images, manually annotated at high resolution.
  • Crack500: Comprises 3368 asphalt crack images, with samples processed into 99x99 pixel patches.
  • CFD: Includes 118 crack images of asphalt and concrete, with ground truth contour lines and noise such as shadows and watermarks.

The paper does not introduce new datasets.

Key Insights and Ideas

  • Texture is Key: The explicit inclusion of a dedicated "Texture Unit" highlights the importance of fine-grained texture and edge information for accurate crack segmentation, especially when cracks blend with the background.
  • Attention for Context: The development of RAM and PFM, incorporating linear attention mechanisms, demonstrates a sophisticated approach to capturing both local details and long-range dependencies, which is critical for understanding the global context of cracks.
  • Hierarchical Fusion: The multistage fusion of features from the Swin-T backbone, the texture unit, and through the CPA, suggests that a comprehensive understanding of crack features requires integrating information at multiple levels of abstraction.
  • Transformer Advantage: The paper reinforces the growing trend of using Transformer architectures (like Swin-T) for vision tasks, showcasing their ability to handle global context and long-range dependencies more effectively than traditional CNNs for complex segmentation problems.

Key Results

FetNet demonstrated superior performance across all four datasets:

  • DeepCrack: Achieved the highest recall (86.4%) and mIoU (82.7%), outperforming the second-best method by 4.2% and 4.1% respectively. It also achieved an F1 score of 89.7%.
  • GAPs384: Showed significant improvements, with FetNet outperforming the second-best method by 6.6% in mIoU.
  • Crack500: Achieved precision of 90.4%, recall of 85.3%, F1 score of 87.9%, and mIoU of 78.6%, surpassing the second-best method by 4.6% in mIoU.
  • CFD: Achieved precision of 98.8%, recall of 85.3%, F1 score of 85.7%, and mIoU of 77.6%, with a notable 5.8% improvement in mIoU over the suboptimal method.
  • Speed: FetNet achieved a segmentation speed of 60.3 FPS on DeepCrack (544x384 pixels), indicating a good balance between accuracy and efficiency.

Qualitatively, FetNet's segmentation results closely resembled the ground truth, with clearer edges and fewer false positives compared to other methods, especially in challenging images with complex textures and noise.

Limitations and Future Work

  • Model Complexity and Resources: The authors acknowledge that FetNet, while effective, has room for improvement in terms of optimizing Swin-T parameters, reducing model complexity, and decreasing computing resource requirements.
  • Data Dependency: The model's performance might still be dependent on the quality and diversity of training data.
  • Future Directions: The paper suggests exploring parameter optimization for Swin-T, reducing computational costs, and investigating the application of FetNet in other domains such as structural detection and bridge maintenance.

Summary updated

Abstract

Structural health monitoring for roads is an important task that supports inspection of transportation infrastructure. This paper explores deep learning techniques for crack detection in road images and proposes an automatic pixel-level semantic road crack image segmentation method based on a Swin transformer. This method employs Swin-T as the backbone network to extract feature information from crack images at various levels and utilizes the texture unit to extract the texture and edge characteristic information of cracks. The refinement attention module (RAM) and panoramic feature module (PFM) then merge these diverse features, ultimately refining the segmentation results. This method is called FetNet. We collect four public real-world datasets and conduct extensive experiments, comparing FetNet with various deep-learning methods. FetNet achieves the highest precision of 90.4%, a recall of 85.3%, an F1 score of 87.9%, and a mean intersection over union of 78.6% on the Crack500 dataset. The experimental results show that the FetNet approach surpasses other advanced models in terms of crack segmentation accuracy and exhibits excellent generalizability for use in complex scenes.
BibTeX
@inproceedings{guo2024detection,
  author = {Maozu Guo and Wenbo Tian and Yang Li and Dong Sui},
  title = {{Detection of Road Crack Images Based on Multistage Feature Fusion and a Texture Awareness Method}},
  booktitle = {Sensors},
  year = {2024},
  doi = {10.3390/s24113268},
}
Figure