Meta-learning Convolutional Neural Architectures for Multi-target Concrete Defect Classification with the COncrete DEfect BRidge IMage Dataset

Martin Mundt, Sagnik Majumder, Sreenivas Murali, Panagiotis Panetsos, Visvanathan Ramesh

Abstract

Recognition of defects in concrete infrastructure, especially in bridges, is a costly and time-consuming crucial first step in the assessment of structural integrity. Large variation in the appearance of concrete material, changing illumination and weather conditions, a variety of possible surface markings, and the possibility for different types of defects to overlap make it a challenging real-world task. In this work, we introduce the novel COncrete DEfect BRidge IMage dataset (CODEBRIM) for multi-target classification of five commonly appearing concrete defects. We investigate and compare two reinforcement learning-based meta-learning approaches, MetaQNN and efficient neural architecture search (ENAS), to find suitable convolutional neural network architectures for this challenging multi-class multi-target task. We show that learned architectures have fewer overall parameters in addition to yielding better multi-target accuracy in comparison to popular neural architectures from the literature evaluated in the context of our application.

Introduction: The Challenge of Real-World Bridge Inspection

Inspecting concrete bridges for structural integrity is currently a slow, expensive, and expert-heavy process. While deep learning has revolutionized image recognition, its application to structural health monitoring has been largely limited to isolated "crack detection."

Figure 6: Examples of overlapping defects and surface markings, such as cracks combined with efflorescence (top) or graffiti (bottom).

Real-world bridge defects are complex and messy. A single area might exhibit a crack, spallation (chipped concrete), and corrosion stains simultaneously. Furthermore, researchers often simply "plug and play" popular models designed for ImageNet (like VGG or AlexNet), which may not be optimized for the specific textures and scales of material defects. This research addresses two gaps: the lack of a multi-target dataset reflecting this overlapping reality and the need for neural architectures tailored to these specific visual patterns.

The CODEBRIM Dataset

The core of this work is the introduction of the COncrete DEfect BRidge IMage (CODEBRIM) dataset, designed to provide a more realistic benchmark for structural health monitoring.

  • Content: 1,590 high-resolution images of 30 unique bridges.
  • Classes: Six mutually non-exclusive categories: crack, spallation, efflorescence (calcium leaching), exposed bars, corrosion (stains), and non-defective background.
  • Acquisition: Images were taken at high resolution (some via UAV) to resolve cracks as small as 0.3mm.
  • Structure: It contains 5,354 annotated defect bounding boxes and 2,506 background boxes. Unlike previous datasets, many boxes contain multiple overlapping defect types.
Figure 4: A representative sample of the CODEBRIM dataset showing the diversity of defects, including exposed reinforcement bars, spallation, and various surface stains.

Methodology: Meta-Learning for Specialized Architectures

To move beyond generic models, the authors employ two different Meta-learning (specifically Neural Architecture Search, or NAS) approaches to "evolve" the best model for the task:

  1. MetaQNN: A reinforcement learning approach where an agent uses Q-learning to pick sequences of layers (convolutions, pooling, etc.) to build a network from scratch.
  2. ENAS (Efficient Neural Architecture Search): A modern approach that speeds up the search by allowing different candidate architectures to share weights, avoiding the need to train every single "child" model from zero.

To handle the multi-target nature of the task, the standard Softmax output was replaced with a Sigmoid function for every class, utilizing binary cross-entropy loss. They also utilized Stochastic Gradient Descent with Warm Restarts (SGDWR) to find better local minima during training.

Key Results and Efficiency

The meta-learned architectures proved to be both more accurate and significantly more efficient than standard baselines:

  • Accuracy: The best meta-learned model (MetaQNN-3) achieved a multi-target test accuracy of 72.19%, outperforming standard baselines like VGG-A (70.45%) and DenseNet-121 (70.77%).
  • Efficiency: The MetaQNN and ENAS models were remarkably lean. For example, MetaQNN-2 achieved 67.45% accuracy with only 1.22 million parameters, whereas VGG-A required 128.79 million parameters to achieve similar results.
  • Depth: While standard models often use dozens or hundreds of layers, the top meta-learned models typically stayed between 6 and 8 layers deep, suggesting that "bigger is not always better" for specialized material recognition.

Conclusion

This work demonstrates that ImageNet performance does not necessarily translate to specialized domains like bridge health monitoring. By introducing the CODEBRIM dataset and utilizing Neural Architecture Search, the authors show that specialized, efficient models can outperform massive, generic architectures. Future work will focus on transitioning from patch-based classification to full-image object detection and integrating these efficient models into UAV-based systems for real-time, autonomous bridge inspection.

Summary updated

Abstract

Recognition of defects in concrete infrastructure, especially in bridges, is a costly and time consuming crucial first step in the assessment of the structural integrity. Large variation in appearance of the concrete material, changing illumination and weather conditions, a variety of possible surface markings as well as the possibility for different types of defects to overlap, make it a challenging real-world task. In this work we introduce the novel COncrete DEfect BRidge IMage dataset (CODEBRIM) for multi-target classification of five commonly appearing concrete defects. We investigate and compare two reinforcement learning based meta-learning approaches, MetaQNN and efficient neural architecture search, to find suitable convolutional neural network architectures for this challenging multi-class multi-target task. We show that learned architectures have fewer overall parameters in addition to yielding better multi-target accuracy in comparison to popular neural architectures from the literature evaluated in the context of our application.
BibTeX
@inproceedings{mundt2019meta,
  author = {Martin Mundt and Sagnik Majumder and Sreenivas Murali and Panagiotis Panetsos and Visvanathan Ramesh},
  title = {{Meta-learning Convolutional Neural Architectures for Multi-target Concrete Defect Classification with the COncrete DEfect BRidge IMage Dataset}},
  booktitle = {CVPR},
  year = {2019},
  url = {https://openreview.net/forum?id=H7V9Tzeu6B},
}
Figure