Industrial Product Surface Anomaly Detection with Realistic Synthetic Anomalies Based on Defect Map Prediction

Tao Peng, Yu Zheng, Lin Zhao, Zheng En-rangSensors 2024 · DOI

TL;DR

This paper tackles the challenge of industrial surface defect detection, where real defect data is scarce. They propose a novel semi-supervised approach that uses realistic synthetic anomalies for training an autoencoder-based reconstruction network and a defect separation network, achieving state-of-the-art results on the MVTec AD dataset.

What Problem Does This Paper Address?

Detecting surface anomalies on industrial products is crucial for maintaining quality and efficiency. However, a major hurdle in applying deep learning for this task is the extreme scarcity of real defect samples. This imbalance makes traditional supervised learning methods prone to overfitting and poor generalization. While anomaly detection methods that rely on reconstruction from normal samples are promising, they can sometimes reconstruct defects too perfectly, leading to missed detections. Furthermore, existing methods for generating synthetic anomalies often lack realism, creating defects that don't accurately reflect real-world scenarios, which can hinder the training of robust detection models.

Methods

The proposed method combines an image reconstruction network with a defect separation network.

  • Realistic Synthetic Anomaly Generation: Unlike previous methods that might create unrealistic anomalies (e.g., placing bottle edges in the middle of an image), this approach focuses on generating more authentic synthetic defects. It involves a three-stage process:
  • Extracting an anomaly source from normal images and identifying foreground objects.
  • Using Perlin noise to generate a random texture and combining it with foreground masks to constrain the anomaly's location to the target object.
  • Blending the anomaly source with the original image using random interpolation to create a final synthesized anomaly image that retains some original image information, making it more realistic and diverse.
  • Image Reconstruction Network: This network is an autoencoder with a deep feature extractor. It's trained to reconstruct normal images while ensuring high-level semantic and low-level visual similarity between the original and reconstructed images. A novel aspect is the use of a deep feature consistency constraint (L_deep loss) alongside visual reconstruction losses (L_vision, combining L2 and SSIM losses). Crucially, the deep feature extractor's parameters are copied from the autoencoder's encoder, rather than being trained directly, to maintain the integrity of extracted features.
  • Defect Separation Network: This network takes the reconstructed image and the synthesized abnormal image as input. It employs a large convolutional kernel (RepLKNet-inspired) design to achieve a wider receptive field, enabling better understanding of global structures and contextual information. This network predicts a defect map, highlighting potential anomaly locations. The loss function for this network uses Focal Loss, which is effective for imbalanced datasets and difficult classification tasks.

The overall loss function combines the reconstruction loss and the defect prediction loss.

Datasets

  • MVTec Anomaly Detection (MVTec AD) Dataset: This is the primary dataset used for both training and evaluation. It's a challenging benchmark containing approximately 5000 real industrial defect images across 15 categories. The dataset provides pixel-level mask annotations for defects.
  • DTD (Describable Textures Dataset): Mentioned as a source for anomaly generation in the DRAEM method, but the proposed method uses the MVTec AD dataset itself for anomaly synthesis.

Key Insights and Ideas

  • Realism in Synthetic Anomalies is Key: The paper strongly emphasizes that generating realistic synthetic anomalies, which retain some original image information and appear on the foreground object, significantly improves the robustness and generalization of anomaly detection models.
  • Deep Feature Consistency Matters: Incorporating a deep feature consistency loss alongside visual reconstruction loss helps the network learn more discriminative features, leading to better reconstruction and, consequently, better anomaly detection.
  • Parameter Copying for Feature Extractors: Using a direct copying strategy for the deep feature extractor's parameters (from the autoencoder's encoder) proved more effective than direct training. This preserves the integrity of features learned from normal data without the risk of them being corrupted by direct training on potentially noisy or imbalanced data.
  • Large Receptive Fields for Context: The use of large convolutional kernels in the defect prediction network allows it to capture broader contextual information, which is vital for distinguishing subtle anomalies from normal variations.

Key Results

  • State-of-the-Art Performance: The proposed method achieved an AUROC score of 99.70% and an average precision (AP) score of 99.87% on the MVTec AD dataset.
  • Superiority over Baselines: The method outperformed existing state-of-the-art algorithms, achieving the highest AUROC in 14 out of 15 categories and surpassing the DRAEM baseline by 31.5% in defect localization (AP).
  • High Accuracy and Speed: The defect detection accuracy reached 98.41%, with an average inference time of 0.041 seconds per sample.
  • Robustness: The method demonstrated exceptional stability on texture-based datasets and regular-shaped objects.

Limitations and Future Work

  • Performance on Certain Data Types: The authors note that the model performs "poorly on several types of data," which they attribute to the realism of their synthetic anomalies posing a greater challenge for anomaly detection.
  • Future Work: Not explicitly detailed in the provided text, but the success of the method suggests potential for application to other industrial inspection tasks and further refinement of synthetic anomaly generation techniques. The paper also highlights the effectiveness of large kernel convolutions and parameter copying, which could be explored in other deep learning architectures.

Summary updated

Abstract

The occurrence of anomalies on the surface of industrial products can lead to issues such as decreased product quality, reduced production efficiency, and safety hazards. Early detection and resolution of these problems are crucial for ensuring the quality and efficiency of production. The key challenge in applying deep learning to surface defect detection of industrial products is the scarcity of defect samples, which will make supervised learning methods unsuitable for surface defect detection problems. Therefore, it is a reasonable solution to use anomaly detection methods to deal with surface defect detection. Among image-based anomaly detection, reconstruction-based methods are the most commonly used. However, reconstruction-based approaches lack the involvement of defect samples in the training process, posing the risk of a perfect reconstruction of defects by the reconstruction network. In this paper, we propose a reconstruction-based defect detection algorithm that addresses these challenges by utilizing more realistic synthetic anomalies for training. Our model focuses on creating authentic synthetic defects and introduces an auto-encoder image reconstruction network with deep feature consistency constraints, as well as a defect separation network with a large receptive field. We conducted experiments on the challenging MVTec anomaly detection dataset and our trained model achieved an AUROC score of 99.70% and an average precision (AP) score of 99.87%. Our method surpasses recently proposed defect detection algorithms, thereby enhancing the accuracy of surface defect detection in industrial products.
BibTeX
@inproceedings{peng2024industrial,
  author = {Tao Peng and Yu Zheng and Lin Zhao and Zheng En-rang},
  title = {{Industrial Product Surface Anomaly Detection with Realistic Synthetic Anomalies Based on Defect Map Prediction}},
  booktitle = {Sensors},
  year = {2024},
  doi = {10.3390/s24010264},
}
Figure