dacl10k: Benchmark for Semantic Bridge Damage Segmentation

Johannes Flotzinger, Philipp J. Rösch, Thomas BramlWACV 2024

TL;DR

This paper introduces "dacl10k," the largest and most diverse dataset for semantic segmentation of bridge defects, comprising over 9,900 real-world images with 18 classes. They also present baseline models and an evaluation framework to establish a benchmark for automated bridge inspection, aiming to improve structural integrity assessment and traffic safety.

What Problem Does This Paper Address?

Bridges are critical infrastructure, but many are aging and showing signs of deterioration due to heavy traffic, weather, and de-icing chemicals. Traditional bridge inspections are manual, time-consuming, and subjective, leading to a growing backlog of structures in poor condition. While automated defect recognition holds great promise for more efficient and reliable inspections, the field lacks comprehensive, diverse datasets for training and benchmarking semantic segmentation models. Existing datasets are often small, lack variety in damage types, or only support binary or single-label segmentation, failing to capture the complex, multi-faceted nature of real-world bridge damage.

Methods

The core contribution is the creation of the dacl10k dataset, a multi-label semantic segmentation benchmark. The dataset was curated from real-world bridge inspections, featuring 9,920 images with pixel-level annotations for 12 types of concrete defects and 6 bridge components. The annotation process involved civil engineering students and an external team, guided by established inspection standards, ensuring a high level of detail and consistency.

For benchmarking, the authors evaluated several state-of-the-art semantic segmentation architectures:

  • DeepLabV3+ and Feature Pyramid Network (FPN) as CNN-based backbones.
  • MobileNetV3-Large, EfficientNet-B2, and EfficientNet-B4 as encoders.
  • SegFormer, a Transformer-based model.

They also explored the use of an auxiliary loss for multi-label classification to improve performance. Models were trained using a combination of Dice and Cross-Entropy loss (or just Dice for SegFormer), with ImageNet pre-trained weights.

Datasets

  • dacl10k: The primary contribution, a new benchmark dataset for multi-label semantic bridge damage segmentation. It contains 9,920 images from over 100 bridges, annotated with 18 classes (12 defects, 6 components). This dataset is significantly larger and more diverse than previous benchmarks.
  • Existing datasets: The paper references CrackSeg9k and S2DS as prior benchmarks in the RCD domain, noting their limitations in scale, class variety, and multi-label capabilities.

Key Insights and Ideas

  • Multi-label segmentation is crucial: Real-world bridge defects often overlap, making multi-label segmentation essential for accurate representation, unlike single-label approaches.
  • Dataset diversity matters: The dacl10k dataset's broad range of damage types, bridge components, and image acquisition conditions (camera types, lighting, resolution) is vital for developing models that generalize well to real-world scenarios.
  • Class similarity poses a challenge: Defects with subtle visual differences (e.g., Weathering vs. Wetspot, Rockpocket vs. Spalling) are difficult for models to distinguish, leading to lower IoU scores.
  • Object classes are easier to segment: Bridge components like Protective Equipment and Bearing, which have distinct visual features, achieve higher IoU scores than many defect classes.
  • Annotation granularity trade-off: While finer annotations are beneficial for defects like cracks (where width is critical), coarser annotations are sufficient for others (like Spalling) where rough measurements are acceptable in practice.

Key Results

  • The best performing model, an FPN architecture with an EfficientNet-B4 encoder and auxiliary loss, achieved a mean Intersection-over-Union (IoU) of 0.414 on the validation split and 0.424 on the test split.
  • Protective Equipment and Bearing showed the highest IoU scores (0.715 and 0.564, respectively), attributed to their distinct visual appearance and good representation.
  • Classes like Washouts/Concrete Corrosion and Restformwork had the lowest IoU scores (0.121 and 0.285, respectively), likely due to visual similarity with other classes or varied appearances.
  • The SegFormer model achieved a respectable mean IoU of 0.400, demonstrating the potential of Transformer architectures in this domain.
  • A qualitative evaluation (Engineer vs. Machine) showed that the best model performed comparably to or better than the average human annotation on simpler samples, but struggled with complex scenes and subtle defect distinctions.

Limitations and Future Work

  • Class imbalance: The dataset exhibits significant class imbalance, with "Background" being overwhelmingly dominant, which can bias model training.
  • Subtle visual differences: Distinguishing between classes with similar visual features (e.g., Weathering, Wetspot, Rockpocket, Spalling) remains a challenge.
  • Finer crack segmentation: For critical defects like cracks, where precise width measurement is important, current annotations might be too coarse. Approaches from weakly-labeled data could be explored.
  • Future work: The authors suggest exploring more sophisticated architectures, hyperparameter tuning, and advanced data augmentation techniques. They also propose investigating methods for finer crack segmentation and addressing the challenges posed by class imbalance and visually similar classes.

Summary updated

BibTeX
@inproceedings{flotzinger2024dacl,
  author = {Johannes Flotzinger and Philipp J. Rösch and Thomas Braml},
  title = {{dacl10k: Benchmark for Semantic Bridge Damage Segmentation}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2024},
}
Figure