Deep Learning-Based Defect Detection Framework for Ultra High Resolution Images of Tunnels

Kisu Lee, Sanghyo Lee, Ha Young KimSustainability 2023 · DOI

TL;DR

This paper introduces a novel framework for detecting defects in ultra-high resolution (UHR) tunnel images, overcoming the challenges of information loss and computational burden associated with traditional downscaling methods. By employing a patchification-based preprocessing and a box smoothing postprocessing approach, the framework significantly boosts defect detection performance, achieving an impressive 77.19 percentage point improvement.

What Problem Does This Paper Address?

Modern tunnel inspection systems generate ultra-high resolution (UHR) images, which are crucial for identifying even minor defects. However, these UHR images pose significant challenges for deep learning models. Standard preprocessing techniques like downscaling lead to a loss of critical information, especially for small defects, thus degrading detection accuracy. Furthermore, training deep learning models directly on UHR images requires substantial computational resources, leading to issues like unstable training and reduced detection rates due to limited batch sizes and increased model complexity. This paper aims to bridge this gap by developing a framework that effectively handles UHR images without sacrificing detail or overwhelming computational resources.

Methods

The proposed framework introduces a multi-stage approach:

  • Preprocessing (Patchification): Instead of downscaling the entire UHR image, the images are divided into smaller, manageable patches. This preserves the original resolution of the defects within each patch, mitigating information loss.
  • Cleansing: A crucial step is introduced to handle "error samples" – patches where a defect's bounding box is split across patch boundaries, potentially leading to misinterpretations during training. This cleansing process uses defined rules based on bounding box edge sharing and size thresholds to filter out these problematic annotations.
  • Sampling: For training, only patches containing defects are used, further optimizing the learning process.
  • Detection: A standard deep learning object detection model, specifically Faster R-CNN, is employed. This model is trained on the patchified images generated during preprocessing.
  • Postprocessing (Box Smoothing): Since detection is performed on patches, a postprocessing step is necessary to reconstruct the results onto the original UHR images. The "box smoothing" method merges overlapping bounding boxes from adjacent patches that belong to the same defect class and satisfy a distance threshold, creating a single, accurate bounding box for the defect on the UHR image. This step is vital for fair performance evaluation.

Datasets

The study utilizes a custom dataset of 1120 UHR images of tunnel linings. These images contain annotations for three primary defect classes: "crack," "crazing," and "delamination." The paper notes a data imbalance among these classes, with cracks being the most frequent. The dataset is split into training, validation, and test sets with a 7:1:2 ratio. The BoxAug method was applied to the training dataset to address the class imbalance.

Key Insights and Ideas

  • Patchification as a Solution to UHR Challenges: The core insight is that breaking down UHR images into patches is a more effective preprocessing strategy than simple downscaling. This approach preserves fine details crucial for detecting small defects while making the images computationally manageable.
  • The Importance of Data Cleansing: The identification and removal of "error samples" arising from patchification are critical. This step significantly improves the robustness of the training process by preventing the model from learning from misaligned or incomplete annotations.
  • Box Smoothing for Accurate UHR-Level Evaluation: Reconstructing patch-level predictions back to the UHR image level using box smoothing is essential for a fair and accurate assessment of the framework's performance. This method ensures that the final evaluation reflects the model's capability on the original, high-resolution images.
  • Generalizability Beyond Tunnels: The framework's design, focusing on UHR image characteristics rather than tunnel-specific features, suggests its potential applicability to defect detection in other domains involving UHR imagery.

Key Results

The proposed framework demonstrated a substantial improvement in defect detection performance. When compared to a traditional downscaling preprocessing method, the framework achieved an average detection rate of 86.71%, a significant leap from the 9.52% achieved by the traditional method. This represents an improvement of approximately 77.19 percentage points (pp). The framework showed particular effectiveness in detecting delamination, which occupies the smallest area in UHR images, highlighting its ability to prevent information loss.

Limitations and Future Work

The authors acknowledge several limitations:

  • Crack Annotation Issues: While the framework improved detection for all classes, the performance gain for cracks was less pronounced than for other defects. This is attributed to the higher number of error samples generated during patchification for cracks, which are often linear and thin. The cleansing step, while effective, could not completely eliminate these errors.
  • Environmental Variability: The study focused on tunnel images. For other structures like bridges and buildings, environmental factors such as weather, shadows, and varying distances can impact image acquisition and potentially affect the framework's performance.

Future work should focus on:

  • Improving the cleansing step to more effectively handle error samples, particularly for linear defects like cracks.
  • Verifying the framework's applicability and performance under diverse environmental conditions for various structures.

Summary updated

Abstract

This study proposes a defect detection framework to improve the performance of deep learning-based detection models for ultra-high resolution (UHR) images generated by tunnel inspection systems. Most of the scanning technologies used in tunnel inspection systems generate UHR images. Defects in real-world images, on the other hand, are noticeably smaller than the image. These characteristics make simple preprocessing applications, such as downscaling, difficult due to information loss. Additionally, when a deep learning model is trained by the UHR images under the limited computational resource for training, problems may occur, including a reduction in object detection rate, unstable training, etc. To address these problems, we propose a framework that includes preprocessing and postprocessing of UHR images related to image patches rather than focusing on deep learning models. Furthermore, it includes a method for supplementing problems according to the format of the data annotation in the preprocessing process. When the proposed framework was applied to the UHR images of a tunnel, the performance of the deep learning-based defect detection model was improved by approximately 77.19 percentage points (pp). Because the proposed framework is for general UHR images, it can effectively recognize damage to general structures other than tunnels. Thus, it is necessary to verify the applicability of the defect detection framework under various conditions in future works.
BibTeX
@inproceedings{lee2023deep,
  author = {Kisu Lee and Sanghyo Lee and Ha Young Kim},
  title = {{Deep Learning-Based Defect Detection Framework for Ultra High Resolution Images of Tunnels}},
  booktitle = {Sustainability},
  year = {2023},
  doi = {10.3390/su15021292},
}
Figure