Deep Learning-Based Fatigue Cracks Detection in Bridge Girders using Feature Pyramid Networks

Jiawei Zhang, Jun Li, Reachsak Ly, Yunyi Liu, Jiangpeng Shu

TL;DR

Researchers have developed a deep learning framework using Feature Pyramid Networks (FPN) to automatically detect and segment fatigue cracks in bridge steel girders. The study demonstrates that splitting high-resolution images into smaller sub-images, rather than resizing them, significantly improves detection accuracy (reaching a mean IoU of 0.78) by preserving the fine, pixel-level details of cracks.

What Problem Does This Paper Address?

Maintaining the safety of aging civil infrastructure, particularly bridges, requires constant monitoring for structural defects like fatigue cracks. Traditionally, this relies on manual visual inspection, which is labor-intensive, expensive, and prone to human error or subjectivity.

While computer vision offers a path toward automation, cracks present a unique challenge: they are often very thin (occupying only a few pixels) and appear against complex, noisy backgrounds like rusted steel or varying lighting conditions. Furthermore, bridge inspection images are typically high-resolution. Feeding these directly into a neural network is computationally prohibitive, but standard "downsizing" techniques often erase the very cracks the system is trying to find.

Methods

The authors propose a semantic segmentation approach based on Feature Pyramid Networks (FPN).

  • Architecture: The FPN uses a "top-down" architecture with lateral connections to build high-level semantic feature maps at all scales. This is ideal for cracks, which can vary significantly in size and orientation.
  • Backbone: They utilized a sophisticated encoder called se_resnext50_32x4d. This combines ResNet (for deep feature extraction), Squeeze-and-Excitation (SE) blocks (an attention mechanism that weights important channels), and ResNeXt (which uses aggregated transformations to improve accuracy without ballooning parameter counts).
  • Preprocessing Comparison: The core of the experiment compared two ways to handle high-resolution input:
  • Resizing: Shrinking the large raw images (e.g., 3264 × 4928) down to smaller dimensions using bilinear interpolation.
  • Splitting: Cutting the raw images into 480 × 640 "sub-images." This preserves the original pixel resolution of the cracks.
  • Training Strategies: They tested models trained only on sub-images containing cracks versus models trained on a mix of crack and background-only sub-images to reduce false positives.

Datasets

The study used a dataset provided by the 1st International Project Competition for Structural Health Monitoring (IPC-SHM, 2020).

  • Total Images: 200 raw images of cracks inside steel box girders.
  • Annotations: 120 images were pixel-level annotated (labeled as "crack" or "non-crack").
  • Resolution: High-resolution files ranging from 3264 × 4928 to 3864 × 5152 pixels.
  • Sub-image Generation: Through the splitting method, the researchers generated 15,861 sub-images, eventually selecting a subset (TS4) of 1,923 crack images and 3,500 background images for optimal training.

Key Insights and Ideas

  • The "Separable" Nature of Cracks: Unlike an object like a car or a face, which requires seeing the "whole" to identify it, a crack can be identified from a small fragment. This makes the splitting method inherently superior for this task, as it allows the model to look at high-resolution details without needing the context of the entire bridge girder.
  • Resizing is a Data Killer: The authors found that resizing images via bilinear interpolation effectively "blurs" thin cracks. Because the interpolation averages neighboring pixels, a one-pixel-wide crack might be mathematically diluted until it is indistinguishable from the background, leading to "shorter" predicted cracks or total misses.
  • Background Matters: Models trained only on sub-images with cracks (Model 3) tended to hallucinate cracks on edges or grooves in the steel. Including background-only sub-images (Model 4) was essential for teaching the model to distinguish between actual fatigue cracks and harmless structural features like weld seams or shadows.

Key Results

  • Top Performance: Model 4 (FPN + Splitting + Background data) was the clear winner, achieving a mean Intersection over Union (mIoU) of 0.78.
  • Resizing vs. Splitting: Models using the resizing method (Models 1 and 2) struggled, with mIoUs hovering around 0.60. They consistently underestimated crack length.
  • Efficiency: While the splitting method involves processing more individual images, it avoids the memory bottlenecks of high-res inputs while providing much higher precision.
  • Qualitative Success: The FPN structure proved highly capable of ignoring "noise" (like handwriting on the girders or rust spots) that typically trips up simpler edge-detection algorithms.

Limitations and Future Work

  • Dataset Size: The study relied on a relatively small set of 120 annotated images. While data augmentation helped, larger and more diverse datasets would improve generalization.
  • Post-Processing: The authors noted that while dilation and thresholding can help "fix" the broken predictions from resized models, these steps currently require manual parameter tuning.
  • Real-world Deployment: The paper focuses on the accuracy of the architecture but does not deeply explore the inference speed required for real-time drone-based or robotic inspections. Future work could involve optimizing the model for "edge" computing devices.

Summary updated

Abstract

For structural health monitoring, continuous and automatic crack detection has been a challenging problem. This study is conducted to propose a framework of automatic crack segmentation from high-resolution images containing crack information about steel box girders of bridges. Considering the multi-scale feature of cracks, convolutional neural network architecture of Feature Pyramid Networks (FPN) for crack detection is proposed. As for input, 120 raw images are processed via two approaches (shrinking the size of images and splitting images into sub-images). Then, models with the proposed structure of FPN for crack detection are developed. The result shows all developed models can automatically detect the cracks at the raw images. By shrinking the images, the computation efficiency is improved without decreasing accuracy. Because of the separable characteristic of crack, models using the splitting method provide more accurate crack segmentations than models using the resizing method. Therefore, for high-resolution images, the FPN structure coupled with the splitting method is an promising solution for the crack segmentation and detection.
BibTeX
@misc{zhang2024deep,
  author = {Jiawei Zhang and Jun Li and Reachsak Ly and Yunyi Liu and Jiangpeng Shu},
  title = {{Deep Learning-Based Fatigue Cracks Detection in Bridge Girders using Feature Pyramid Networks}},
  year = {2024},
  eprint = {2410.21175},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2410.21175},
}
Figure