3D Modeling and Automated Measurement of Concrete Cracks via Segment Anything Refinement and Visual Inertial LiDAR Fusion

Pengru Deng, Jiapeng Yao, Chun Li, Su Wang, Xinrun Li, Varun Ojha, Xuhui HearXiv 2025 · DOI · arXiv

TL;DR

This paper introduces a novel framework that combines a foundation vision model (SAM) with Visual-Inertial-LiDAR SLAM to automate the 3D mapping and sub-millimeter measurement of concrete cracks. By bridging coarse deep-learning segmentation with automated prompt refinement and fusing it with precise 3D point clouds, the system achieves highly accurate, real-world scale crack quantification without manual post-processing.

What Problem Does This Paper Address?

Regular inspection of infrastructure like bridges and tunnels is critical for structural safety, with surface cracks serving as primary indicators of structural degradation. However, current AI-based crack inspection methods suffer from two major bottlenecks:

  1. Poor Generalization (Domain Shift): Standard deep learning models trained on specific datasets fail when deployed in new environments with different lighting, surface textures, stains, or structural clutter.
  2. Decoupled 3D Quantification: Traditional methods either measure cracks in 2D image space (lacking true 3D spatial context and assuming flat surfaces) or use Structure-from-Motion (SfM) photogrammetry, which lacks absolute scale calibration and struggles with real-time processing. Furthermore, high-resolution cameras required to capture tiny cracks operate at low frame rates, causing severe spatial misalignment when fused with high-rate LiDAR data.

This paper tackles these challenges by creating a unified pipeline that reliably segments cracks in unseen environments and maps them into a high-density, semantically enriched 3D point cloud for automated, real-world scale measurement.

Methods

The proposed methodology is split into two primary stages: Crack Identification and Crack Measurement.

Overview of the proposed crack modeling & automated measurement methodology

1. Generalized Concrete Crack Segmentation

Instead of relying solely on a task-specific model or manual prompts for the Segment Anything Model (SAM), the authors design a hybrid approach:

  • Coarse Segmentation: A base DeepLabv3+ model generates initial, potentially fragmented crack masks.
  • Crack-Aware Prompt Generation: A Euclidean Distance Transform (EDT) extracts the skeleton of these coarse masks to automatically sample optimal point prompts for SAM.
  • Quality Assessment Module: To prevent SAM from over-segmenting or "leaking" into non-crack regions, an EDT-based width consistency check automatically rejects poor refinements and falls back to the baseline mask if a threshold is exceeded.
Overview of the proposed segmentation refinement via SAM framework

2. 3D Structure Reconstruction and Multi-Modal Fusion

To bypass the limitations of low-frame-rate cameras, the system utilizes a robust LiDAR-Inertial Odometry (LIO) framework (FastLIO2) for baseline trajectory estimation.

  • Multi-Sensor Calibration: The camera, LiDAR, and IMU are precisely calibrated using a target-less, attention-based graph matching approach.
  • Pose Interpolation: Camera poses are smoothly interpolated from the high-frequency LIO trajectory using Spherical Linear Interpolation (Slerp), eliminating temporal synchronization errors.
  • Crack-Oriented View Weighting: A novel scoring function prioritizes image frames based on the optimal viewing angle and an ideal observation range (~2 meters) to project 2D crack semantics onto the 3D point cloud without distortion.

3. Automated 3D Measurement

Once the 3D semantic point cloud is generated, the system automatically fits a local plane to the crack zone. Using Inverse Perspective Mapping (IPM), it traces the left and right crack boundaries in 3D space and computes the true Euclidean distance between them, enabling automated sub-millimeter width estimation.

Datasets

The authors validated their framework using a two-tiered dataset strategy:

  • Pre-training Pool: 10 diverse, open-source crack datasets totaling 9,260 RGB images (including Ceramic-Cracks, CFD, Crack500, CrackTree200, DeepCrack, GAPS, Masonry, Rissbilder, Volker, and CCSS-DATA).
  • Unseen Testing Dataset: 150 high-resolution (4096×3000) images captured via a custom handheld device on real-world concrete blocks and pedestrian walkway slabs. This dataset included severe real-world interferences like shadows, stains, and reflective tags.

Key Insights and Ideas

  • Small Models as Prompt Generators: Rather than trying to train a small model to be perfectly accurate across all domains, it is far more effective to use it as a "coarse feature detector" to guide a powerful foundation model like SAM.
  • Safety Rails for Foundation Models: Foundation models can exhibit unstable behaviors (like excessive thickening) in complex scenes. Introducing a simple geometric prior—the EDT width ratio constraint—acts as a highly effective safety rail to maintain structural topology.
  • Decoupling Camera Tracking from Trajectory Estimation: By relying on LiDAR-Inertial data for the heavy lifting of spatial tracking and interpolating the camera poses post-hoc, the system successfully integrates high-resolution, low-frame-rate imagery without the tracking failures typical of Visual-Inertial Odometry (VIO).

Key Results

  • Zero-Shot Generalization: The SAM-refinement pipeline achieved a 6% to 7.8% improvement in Intersection over Union (IoU) under zero-shot settings compared to standalone task-specific models.
  • Sub-Millimeter Precision: The automated 3D measurement pipeline achieved a mean absolute error (MAE) of just 0.09 mm for crack width estimation.
  • High-Fidelity Reconstruction: The framework generated exceptionally dense point clouds (3,173,144 pts/m²), vastly outperforming standard SfM (165,208 pts/m²) and traditional LOAM (1,471,886 pts/m²), while reducing geometric dimension mean error to a mere 0.13%.

Limitations and Future Work

Despite its high accuracy, the authors note several areas for improvement:

  • Distance Constraints: The sub-millimeter accuracy is highly distance-dependent. At 0.3 meters, the resolution is 0.069 mm/pixel, but it degrades to 0.113 mm/pixel at 0.5 meters. Future work will focus on adaptive scanning path planning to maintain optimal distances.
  • Platform Portability: The current system is handheld. Transitioning the payload to Unmanned Aerial Vehicles (UAVs) is required for large-scale, automated infrastructure monitoring.
  • Missing Metrics: The framework currently quantifies crack width and spatial position but does not explicitly calculate total crack length or track crack propagation over time, which are key targets for future iterations.

Summary updated

Abstract

Visual-Spatial Systems has become increasingly essential in concrete crack inspection. However, existing methods often lacks adaptability to diverse scenarios, exhibits limited robustness in image-based approaches, and struggles with curved or complex geometries. To address these limitations, an innovative framework for two-dimensional (2D) crack detection, three-dimensional (3D) reconstruction, and 3D automatic crack measurement was proposed by integrating computer vision technologies and multi-modal Simultaneous localization and mapping (SLAM) in this study. Firstly, building on a base DeepLabv3+ segmentation model, and incorporating specific refinements utilizing foundation model Segment Anything Model (SAM), we developed a crack segmentation method with strong generalization across unfamiliar scenarios, enabling the generation of precise 2D crack masks. To enhance the accuracy and robustness of 3D reconstruction, Light Detection and Ranging (LiDAR) point clouds were utilized together with image data and segmentation masks. By leveraging both image- and LiDAR-SLAM, we developed a multi-frame and multi-modal fusion framework that produces dense, colorized point clouds, effectively capturing crack semantics at a 3D real-world scale. Furthermore, the crack geometric attributions were measured automatically and directly within 3D dense point cloud space, surpassing the limitations of conventional 2D image-based measurements. This advancement makes the method suitable for structural components with curved and complex 3D geometries. Experimental results across various concrete structures highlight the significant improvements and unique advantages of the proposed method, demonstrating its effectiveness, accuracy, and robustness in real-world applications.
BibTeX
@misc{deng2025d,
  author = {Pengru Deng and Jiapeng Yao and Chun Li and Su Wang and Xinrun Li and Varun Ojha and Xuhui He},
  title = {{3D Modeling and Automated Measurement of Concrete Cracks via Segment Anything Refinement and Visual Inertial LiDAR Fusion}},
  year = {2025},
  eprint = {2501.09203},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2501.09203},
}

Loading related content...

Figure