Topology-aware Mamba for Crack Segmentation in Structures

Xin Zuo, Yu Sheng, Jifeng Shen, Yongwei ShanAutomation in Construction 2024 · DOI · arXiv · OpenReview

TL;DR

This paper introduces CrackMamba, a novel deep learning model that leverages the Mamba architecture for efficient and accurate crack segmentation in structural health monitoring. By proposing a "Snake Scan" module and a specialized SCVSS block, CrackMamba effectively captures the complex, serpentine nature of cracks, achieving state-of-the-art performance on crack segmentation benchmarks.

What Problem Does This Paper Address?

Accurate and efficient crack segmentation is crucial for structural health monitoring to ensure the safety and integrity of infrastructure. Traditional methods like thresholding and edge detection struggle with complex backgrounds, noise, and illumination variations. While deep learning models like CNNs and Vision Transformers (ViTs) have shown promise, CNNs have limited receptive fields, and ViTs are computationally expensive. Existing approaches often fail to adequately capture the inherently serpentine and complex patterns of crack development, leading to suboptimal segmentation.

Methods

CrackMamba builds upon the VMambaV2 architecture, utilizing it as an encoder with pre-trained ImageNet-1k weights. A key innovation is the Snake Scan module, which reorders image patches into serpentine sequences to better align with crack morphology, enhancing feature extraction. This contrasts with traditional "Cross Scan" methods that process images row-by-row and column-by-column.

The model also introduces the three-branch Snake Conv VSS (SCVSS) block. This block comprises:

  • The original VSS block (from VMamba) to retain horizontal and vertical feature extraction.
  • The SnakeVSS block, which incorporates the Snake Scan module for a more topology-aware sequence processing.
  • A convolutional branch to capture local feature information.

These branches are fused using spatial and channel attention mechanisms. The decoder is a standard U-Net-like structure with upsampling layers and deep supervision.

Datasets

The paper uses the following datasets:

  • CrackSeg9K: A re-screened and augmented dataset comprising 8751 images from various crack datasets, featuring diverse crack morphologies.
  • SewerCrack: A private dataset of 582 images extracted from CCTV videos of sewer pipes, representing a complex environment with significant background noise.
  • CHASE_DB1: A retinal vessel segmentation dataset (28 images) used to evaluate the model's generalization capabilities on crack-like structures.

Key Insights and Ideas

  • Topology-aware scanning is crucial for crack segmentation: The serpentine nature of cracks is not well-captured by standard row/column scanning. The Snake Scan method directly addresses this by creating sequences that mimic crack trajectories.
  • Mamba's efficiency and long-range dependency are well-suited for crack segmentation: The paper demonstrates that Mamba-based models can achieve high performance with fewer parameters than Transformer-based models, making them more practical for real-world applications.
  • Multi-branch fusion enhances feature extraction: Combining the strengths of sequential processing (SnakeVSS), traditional spatial processing (VSS), and local feature extraction (convolutional branch) within the SCVSS block leads to more robust and detailed crack segmentation.
  • Generalization to similar structures: The model's strong performance on retinal vessel segmentation suggests its potential for other applications involving fine, linear, or serpentine structures.

Key Results

  • CrackMamba achieves state-of-the-art (SOTA) performance on the CrackSeg9k and SewerCrack datasets, outperforming CNNs (UNet, DeepLabV3) and Transformer-based models (SwinUNet, TransUNet, Swin-UMamba).
  • On the reconstructed CrackSeg9k dataset, CrackMamba achieved an mIoU of 81.75% and an F1-score of 79.32%.
  • On the SewerCrack dataset, it achieved an mIoU of 70.47% and an F1-score of 58.86%.
  • On the CHASE_DB1 dataset, CrackMamba achieved an IoU of 97.38%, significantly surpassing other models.
  • The model demonstrates a lower parameter count compared to many Transformer-based models while achieving superior performance.

Limitations and Future Work

  • Ground truth inaccuracies: The paper acknowledges that errors in ground truth labeling can affect model performance and plans to address this by reviewing and re-annotating datasets.
  • Similarity between crack and background pixels: Distinguishing fine cracks from similar background textures remains a challenge. Future work will explore deep texture information extraction methods (e.g., wavelet variations).
  • Complex backgrounds: Segmentation in highly complex environments (like sewer pipes) is challenging. The authors plan to expand their sewer pipe dataset and incorporate data from diverse environments to improve generalization.
  • Image-specific crack ordering: Current methods rearrange features generically. Future work will focus on developing image-specific crack feature ordering methods for greater flexibility and adaptability.
  • Public release of dataset: The authors are considering releasing their crack segmentation dataset, which includes more complex contexts than SewerCrack.

Summary updated

Abstract

CrackMamba, a Mamba-based model, is designed for efficient and accurate crack segmentation for monitoring the structural health of infrastructure. Traditional Convolutional Neural Network (CNN) models struggle with limited receptive fields, and while Vision Transformers (ViT) improve segmentation accuracy, they are computationally intensive. CrackMamba addresses these challenges by utilizing the VMambaV2 with pre-trained ImageNet-1k weights as the encoder and a newly designed decoder for better performance. To handle the random and complex nature of crack development, a Snake Scan module is proposed to reshape crack feature sequences, enhancing feature extraction. Additionally, the three-branch Snake Conv VSS (SCVSS) block is proposed to target cracks more effectively. Experiments show that CrackMamba achieves state-of-the-art (SOTA) performance on the CrackSeg9k and SewerCrack datasets, and demonstrates competitive performance on the retinal vessel segmentation dataset CHASE\underline{~}DB1, highlighting its generalization capability. The code is publicly available at: {https://github.com/shengyu27/CrackMamba.}
BibTeX
@inproceedings{zuo2024topology,
  author = {Xin Zuo and Yu Sheng and Jifeng Shen and Yongwei Shan},
  title = {{Topology-aware Mamba for Crack Segmentation in Structures}},
  booktitle = {Automation in Construction},
  year = {2024},
  doi = {10.1016/j.autcon.2024.105845},
}
Figure