Automated Quality Control of Vacuum Insulated Glazing by Convolutional Neural Network Image Classification

Henrik Riedel, Sleheddine Mokdad, Isabell Schulz, Cenk Kocer, Philipp L. Rosendahl, Jens Schneider, Michael A. Kraus, Michael DrassarXiv 2021 · DOI · arXiv · OpenReview

TL;DR

This paper introduces VDCNet, a novel deep learning model for automatically detecting surface cracks in Vacuum Insulated Glazing (VIG) units, a critical step for ensuring their long-term durability. VDCNet significantly outperforms existing state-of-the-art ResNet models in accuracy, speed, and reliability, achieving a perfect 100% AUC and precision at 100% recall on their test dataset.

What Problem Does This Paper Address?

Vacuum Insulated Glazing (VIG) offers superior thermal insulation in a slim profile, making it an attractive window technology. However, tiny pillars used to maintain the vacuum gap can cause microscopic cracks on the glass surface during manufacturing or handling. These cracks, though not immediately catastrophic, can compromise the VIG unit's lifespan and performance over its intended 30+ year service life. Manually inspecting these numerous, tiny cracks is time-consuming and prone to human error. Therefore, an automated, high-precision damage detection system is crucial for quality control and ensuring the reliability of VIG windows.

Methods

The core of this work is the development of a custom Convolutional Neural Network (CNN) called VDCNet, built upon the principles of Residual Networks (ResNets). Unlike standard ResNets, VDCNet concatenates skip connections rather than adding them, allowing each layer to process both filtered and unfiltered data from previous layers. This architecture was specifically designed to be less deep than typical ResNets, as the VIG crack detection task requires focusing on smaller details.

To prepare the data, the researchers employed a two-stage approach:

  1. Object Detection: Faster R-CNN was used to automatically locate and crop the pillars from the raw images, ensuring the relevant area for damage analysis was captured.
  2. Image Preprocessing & Augmentation: Images were cropped to focus on the pillar and its immediate surroundings, then divided into smaller quadrants to increase the dataset size. Data augmentation techniques like random rotation, horizontal/vertical flips, random brightness adjustments, and random pixel overwriting were applied to improve generalization and prevent overfitting.

For understanding the model's decision-making, they utilized Explainable AI (XAI) techniques, specifically Grad-CAM and Score-CAM, to visualize which parts of the image the VDCNet focused on when making a classification.

Datasets

The study utilized a custom dataset of VIG pillar images. The images were captured using a microscope with 1,000x magnification and a 1,920x1,080 resolution sensor. The dataset was manually annotated by human experts into two classes: "undamaged" and "damaged." Approximately 300 images were annotated for the object detection model, and a total of 1,267 image quadrants were used for training and testing the classification model (90% for training, 10% for testing). No pre-existing public datasets for VIG crack detection were used.

Key Insights and Ideas

  • Concatenation vs. Addition in ResNets: The VDCNet's novel approach of concatenating skip connections instead of adding them proved beneficial for this specific task, potentially allowing for richer feature propagation.
  • XAI for Trust and Debugging: Employing Grad-CAM and Score-CAM was crucial not only for understanding why the model made a decision but also for early-stage debugging and validating the model's focus on actual cracks rather than mere brightness variations.
  • Optimized Architecture for Specific Task: A less deep network (VDCNet with 20 convolution layers vs. ResNet50V2's 49) was sufficient and even superior for this focused image classification task, highlighting the importance of tailoring network depth to the problem.
  • Data Augmentation is Key for Limited Data: Given the custom nature of the dataset, robust data augmentation was essential for achieving high generalization performance.
  • Perfect Performance is Achievable: The paper demonstrates that with a well-designed model and appropriate data handling, near-perfect performance (100% AUC and precision at 100% recall) is attainable for this critical quality control task.

Key Results

  • Classification Accuracy: VDCNet achieved a flawless 100% Area Under the Curve (AUC) for the Receiver Operating Characteristic (ROC) and 100% precision at 100% recall on the test dataset.
  • Outperformance of ResNets: VDCNet consistently outperformed ResNet50V2, ResNet101V2, and ResNet152V2 across multiple metrics, including:
  • Fewer training epochs (49 vs. 67-75).
  • Significantly fewer false negative (FN) classifications (4.39 times less frequent) and false positive (FP) classifications (34.5 times less frequent) under specific threshold conditions.
  • Higher precision at 100% recall.
  • XAI Insights: Grad-CAM visualizations showed that VDCNet focused on the actual crack features, whereas ResNet50V2's visualizations were less precise and sometimes focused on irrelevant areas.
  • Efficiency: The VDCNet required significantly fewer training epochs compared to the ResNet models.

Limitations and Future Work

  • Manual Data Preparation: The initial data acquisition, preprocessing, and labeling were performed manually, which is time-consuming. The authors suggest that an automated hardware system (e.g., a sliding rail) is necessary for industrial deployment.
  • Limited Dataset Size: While data augmentation helped, the dataset was custom-built and relatively small. The authors note that further training could potentially include learning the fracture process itself.
  • Future Work:
    • Integrating VDCNet with automated hardware for efficient image acquisition.
  • Developing object detection models to selectively capture pillar images, further reducing processing time.
  • Linking the VDCNet's damage detection capabilities with mechanical models of glass failure to predict crack propagation and inform damage mitigation strategies.
  • Exploring the use of VDCNet for identifying the cause of cracks, not just their presence.

Summary updated

Abstract

Vacuum Insulated Glazing (VIG) is a highly thermally insulating window technology, which boasts an extremely thin profile and lower weight as compared to gas-filled insulated glazing units of equivalent performance. The VIG is a double-pane configuration with a submillimeter vacuum gap between the panes and therefore under constant atmospheric pressure over their service life. Small pillars are positioned between the panes to maintain the gap, which can damage the glass reducing the lifetime of the VIG unit. To efficiently assess any surface damage on the glass, an automated damage detection system is highly desirable. For the purpose of classifying the damage, we have developed, trained, and tested a deep learning computer vision system using convolutional neural networks. The classification model flawlessly classified the test dataset with an area under the curve (AUC) for the receiver operating characteristic (ROC) of 100%. We have automatically cropped the images down to their relevant information by using Faster-RCNN to locate the position of the pillars. We employ the state-of-the-art methods Grad-CAM and Score-CAM of explainable Artificial Intelligence (XAI) to provide an understanding of the internal mechanisms and were able to show that our classifier outperforms ResNet50V2 for identification of crack locations and geometry. The proposed methods can therefore be used to detect systematic defects even without large amounts of training data. Further analyses of our model's predictive capabilities demonstrates its superiority over state-of-the-art models (ResNet50V2, ResNet101V2 and ResNet152V2) in terms of convergence speed, accuracy, precision at 100% recall and AUC for ROC.
BibTeX
@misc{riedel2021automated,
  author = {Henrik Riedel and Sleheddine Mokdad and Isabell Schulz and Cenk Kocer and Philipp L. Rosendahl and Jens Schneider and Michael A. Kraus and Michael Drass},
  title = {{Automated Quality Control of Vacuum Insulated Glazing by Convolutional Neural Network Image Classification}},
  year = {2021},
  eprint = {2110.08079},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2110.08079},
}
Figure