A Generic Semi-Supervised Deep Learning-Based Approach for Automated Surface Inspection

Xiaoqing Zheng, Hongcheng Wang, Jie Chen, Yaguang Kong, Song ZhengIEEE Access 2020 · OpenReview

Abstract

Automated surface inspection (ASI) is critical to quality control in industrial manufacturing processes. Recent advances in deep learning have produced new ASI methods that automatically learn high-level features from training samples while being robust to changes and capable of detecting different types of surfaces and defects. However, they usually rely heavily on manpower to collect and label training samples. In this paper, a generic semi-supervised deep learning-based approach for ASI that requires a small quantity of labeled training data is proposed. While the approach follows the MixMatch rules to conduct sophisticated data augmentation, we introduce a new loss function calculation method and propose a new convolutional neural network based on a residual structure to achieve accurate defect detection. An experiment on two public datasets (DAGM and NEU) and one industrial dataset (CCL) is carried out. The proposed method achieves the best performance in all comparisons, demonstrating that it can achieve good performance with few labeled training samples.

Introduction

Automated Surface Inspection (ASI) is a cornerstone of modern industrial quality control, used to detect defects like scratches, pits, or stains on manufactured goods. While deep learning, specifically Convolutional Neural Networks (CNNs), has revolutionized this field, these models are notoriously "data-hungry," requiring thousands of manually labeled images to perform reliably. In an industrial setting, collecting and labeling these images is a major bottleneck, requiring significant man-hours from domain experts and slowing down the deployment of new inspection lines. This paper addresses the "labeling crisis" by proposing a semi-supervised learning (SSL) approach that leverages large amounts of unlabeled data alongside a tiny set of labeled samples to reach—and sometimes exceed—the performance of fully supervised models.

Methodology

The proposed approach builds upon the MixMatch framework but introduces several key modifications tailored for surface inspection.

Semi-Supervised Framework

The core of the system is a pipeline that processes both labeled and unlabeled samples. For unlabeled data, the model performs "label guessing" by averaging predictions from multiple augmented versions of the same image, followed by a "sharpening" step to reduce entropy. These guessed labels are then combined with labeled data using the MixUp technique to create a robust training set.

Overview of the proposed semi-supervised learning framework based on MixMatch.

Specialized 43-Layer Residual CNN

The authors designed a custom architecture based on ResNet principles. It utilizes 43 layers, stacking two-layer residual blocks to balance network depth and width. This structure helps the model converge more easily and prevents the "degradation problem" common in very deep networks.

Structure of the residual block used in the 43-layer CNN.

Data Augmentation and Hybrid Loss

Beyond standard flips or crops, the authors employ "Cutout" technology, which randomly masks square regions of an image. This forces the network to learn from the remaining context, making it more robust to various defect shapes and locations.

A novel hybrid loss function is introduced, where the total loss is a weighted sum of a supervised loss ($L_S$) and an unsupervised loss ($L_U$). The unsupervised loss is a unique blend of Mean Square Error (MSE) and Kullback-Leibler (KL) divergence. While MSE helps with general distribution matching, KL divergence specifically penalizes differences in the probability distributions of predicted labels, leading to better refinement of the model's "guesses" on unlabeled data.

Experimental Evaluation

The model was evaluated on three distinct datasets:

  • DAGM Texture Dataset: A synthetic benchmark for defect detection on statistically textured surfaces.
  • NEU Steel Dataset: Real-world images of six types of typical surface defects on hot-rolled steel strips.
  • CCL Industrial Dataset: A challenging real-world dataset from a copper clad laminate factory, categorized into severe and non-severe defects.

Key Results

The proposed method consistently outperformed established benchmarks:

  • NEU Dataset: Achieved a perfect 100% accuracy using only 30% of the labeled data (378 samples).
  • DAGM Dataset: Reached 99.83% accuracy with 70% labels, outperforming several supervised CNN benchmarks.
  • CCL Industrial Dataset: Achieved 95.67% accuracy, significantly beating "Bad GAN" (90.98%) and even supervised ResNet-101 (94.64%) trained on the full dataset.
  • Efficiency: Inference times range from 7ms to 44ms per image, making it suitable for real-time production.

Conclusion

This research demonstrates that unlabeled data provides a strong regularization effect when processed through a sophisticated SSL pipeline. Combining MSE and KL divergence for unsupervised loss significantly improves performance, suggesting that defect detection requires attention to both absolute error and the shape of probability distributions. Future work will focus on improving computational efficiency for higher-speed lines and enhancing the stability of the SSL training process to ensure consistency across diverse industrial materials.

Summary updated

Abstract

Automated surface inspection (ASI) is critical to quality control in industrial manufacturing processes. Recent advances in deep learning have produced new ASI methods that automatically learn high-level features from training samples while being robust to changes and capable of detecting different types of surfaces and defects. However, they usually rely heavily on manpower to collect and label training samples. In this paper, a generic semi-supervised deep learning-based approach for ASI that requires a small quantity of labeled training data is proposed. While the approach follows the MixMatch rules to conduct sophisticated data augmentation, we introduce a new loss function calculation method and propose a new convolutional neural network based on a residual structure to achieve accurate defect detection. An experiment on two public datasets (DAGM and NEU) and one industrial dataset (CCL) is carried out. For public datasets, the experimental results are compared against several best benchmarks in the literature. For the industrial dataset, the results are compared against deep learning methods based on benchmark neural networks. The proposed method achieves the best performance in all comparisons. In addition, a comparative experiment of model performance given a different number of labeled samples is conducted, demonstrating that the proposed method can achieve good performance with few labeled training samples.
BibTeX
@inproceedings{zheng2020generic,
  author = {Xiaoqing Zheng and Hongcheng Wang and Jie Chen and Yaguang Kong and Song Zheng},
  title = {{A Generic Semi-Supervised Deep Learning-Based Approach for Automated Surface Inspection}},
  booktitle = {IEEE Access},
  year = {2020},
  url = {https://openreview.net/forum?id=8vIKmqRLZ8},
}
Figure