TL;DR
This paper introduces a novel framework that combines a foundation vision model (SAM) with Visual-Inertial-LiDAR SLAM to automate the 3D mapping and sub-millimeter measurement of concrete cracks. By bridging coarse deep-learning segmentation with automated prompt refinement and fusing it with precise 3D point clouds, the system achieves highly accurate, real-world scale crack quantification without manual post-processing.
What Problem Does This Paper Address?
Regular inspection of infrastructure like bridges and tunnels is critical for structural safety, with surface cracks serving as primary indicators of structural degradation. However, current AI-based crack inspection methods suffer from two major bottlenecks:
- Poor Generalization (Domain Shift): Standard deep learning models trained on specific datasets fail when deployed in new environments with different lighting, surface textures, stains, or structural clutter.
- Decoupled 3D Quantification: Traditional methods either measure cracks in 2D image space (lacking true 3D spatial context and assuming flat surfaces) or use Structure-from-Motion (SfM) photogrammetry, which lacks absolute scale calibration and struggles with real-time processing. Furthermore, high-resolution cameras required to capture tiny cracks operate at low frame rates, causing severe spatial misalignment when fused with high-rate LiDAR data.
This paper tackles these challenges by creating a unified pipeline that reliably segments cracks in unseen environments and maps them into a high-density, semantically enriched 3D point cloud for automated, real-world scale measurement.
Methods
The proposed methodology is split into two primary stages: Crack Identification and Crack Measurement.
1. Generalized Concrete Crack Segmentation
Instead of relying solely on a task-specific model or manual prompts for the Segment Anything Model (SAM), the authors design a hybrid approach:
- Coarse Segmentation: A base DeepLabv3+ model generates initial, potentially fragmented crack masks.
- Crack-Aware Prompt Generation: A Euclidean Distance Transform (EDT) extracts the skeleton of these coarse masks to automatically sample optimal point prompts for SAM.
- Quality Assessment Module: To prevent SAM from over-segmenting or "leaking" into non-crack regions, an EDT-based width consistency check automatically rejects poor refinements and falls back to the baseline mask if a threshold is exceeded.
2. 3D Structure Reconstruction and Multi-Modal Fusion
To bypass the limitations of low-frame-rate cameras, the system utilizes a robust LiDAR-Inertial Odometry (LIO) framework (FastLIO2) for baseline trajectory estimation.
- Multi-Sensor Calibration: The camera, LiDAR, and IMU are precisely calibrated using a target-less, attention-based graph matching approach.
- Pose Interpolation: Camera poses are smoothly interpolated from the high-frequency LIO trajectory using Spherical Linear Interpolation (Slerp), eliminating temporal synchronization errors.
- Crack-Oriented View Weighting: A novel scoring function prioritizes image frames based on the optimal viewing angle and an ideal observation range (~2 meters) to project 2D crack semantics onto the 3D point cloud without distortion.
3. Automated 3D Measurement
Once the 3D semantic point cloud is generated, the system automatically fits a local plane to the crack zone. Using Inverse Perspective Mapping (IPM), it traces the left and right crack boundaries in 3D space and computes the true Euclidean distance between them, enabling automated sub-millimeter width estimation.
Datasets
The authors validated their framework using a two-tiered dataset strategy:
- Pre-training Pool: 10 diverse, open-source crack datasets totaling 9,260 RGB images (including Ceramic-Cracks, CFD, Crack500, CrackTree200, DeepCrack, GAPS, Masonry, Rissbilder, Volker, and CCSS-DATA).
- Unseen Testing Dataset: 150 high-resolution (4096×3000) images captured via a custom handheld device on real-world concrete blocks and pedestrian walkway slabs. This dataset included severe real-world interferences like shadows, stains, and reflective tags.
Key Insights and Ideas
- Small Models as Prompt Generators: Rather than trying to train a small model to be perfectly accurate across all domains, it is far more effective to use it as a "coarse feature detector" to guide a powerful foundation model like SAM.
- Safety Rails for Foundation Models: Foundation models can exhibit unstable behaviors (like excessive thickening) in complex scenes. Introducing a simple geometric prior—the EDT width ratio constraint—acts as a highly effective safety rail to maintain structural topology.
- Decoupling Camera Tracking from Trajectory Estimation: By relying on LiDAR-Inertial data for the heavy lifting of spatial tracking and interpolating the camera poses post-hoc, the system successfully integrates high-resolution, low-frame-rate imagery without the tracking failures typical of Visual-Inertial Odometry (VIO).
Key Results
- Zero-Shot Generalization: The SAM-refinement pipeline achieved a 6% to 7.8% improvement in Intersection over Union (IoU) under zero-shot settings compared to standalone task-specific models.
- Sub-Millimeter Precision: The automated 3D measurement pipeline achieved a mean absolute error (MAE) of just 0.09 mm for crack width estimation.
- High-Fidelity Reconstruction: The framework generated exceptionally dense point clouds (3,173,144 pts/m²), vastly outperforming standard SfM (165,208 pts/m²) and traditional LOAM (1,471,886 pts/m²), while reducing geometric dimension mean error to a mere 0.13%.
Limitations and Future Work
Despite its high accuracy, the authors note several areas for improvement:
- Distance Constraints: The sub-millimeter accuracy is highly distance-dependent. At 0.3 meters, the resolution is 0.069 mm/pixel, but it degrades to 0.113 mm/pixel at 0.5 meters. Future work will focus on adaptive scanning path planning to maintain optimal distances.
- Platform Portability: The current system is handheld. Transitioning the payload to Unmanned Aerial Vehicles (UAVs) is required for large-scale, automated infrastructure monitoring.
- Missing Metrics: The framework currently quantifies crack width and spatial position but does not explicitly calculate total crack length or track crack propagation over time, which are key targets for future iterations.
Summary updated