Defect-GAN: High-Fidelity Defect Synthesis for Automated Defect Inspection

Gongjie Zhang, Kaiwen Cui, Tzu-Yi Hung, Shijian Lu

Abstract

Defect-GAN is a novel deep learning framework designed to address the critical "data scarcity" problem in industrial visual inspection. By mimicking the physical processes of defacement and restoration, Defect-GAN synthesizes high-fidelity, diverse defect samples. This approach generates realistic training data that significantly enhances the accuracy and robustness of automated defect detection models, bridging the gap between rare real-world defects and the data requirements of deep neural networks.

Introduction

Automated visual inspection is essential for modern manufacturing and infrastructure maintenance, such as identifying cracks in bridges or flaws in machinery. However, training deep neural networks for these tasks is hindered by the rarity of defects. In typical production environments, the vast majority of samples are "normal," making the collection of large, labeled datasets for rare defects expensive and often impossible.

Existing data augmentation techniques, including standard Generative Adversarial Networks (GANs), often produce blurry results or alter background textures excessively, rendering the synthetic data ineffective for training. Defect-GAN addresses this by "painting" realistic defects onto normal surfaces while preserving the integrity of the original background.

Overview of the Defect-GAN framework for synthesizing defect samples to train inspection networks.

Methodology

The core of Defect-GAN lies in its ability to model the relationship between clean and damaged surfaces through several innovative architectural choices:

Defacement and Restoration Cycles

The network employs a cyclic learning process. The "Defacement" phase generates defects on normal surfaces, while the "Restoration" phase learns to remove them. This dual-direction learning helps the model understand the nuanced differences between a clean surface and a defective one.

Compositional Layer-based Architecture

Unlike standard GANs that generate entire images from scratch, Defect-GAN generates a defect foreground and a spatial distribution map. These components are then composed onto the original normal image. This ensures that the background texture—such as the specific grain of concrete or metal—remains untouched, allowing the defect to appear naturally integrated.

The compositional layer-based architecture of Defect-GAN, featuring defacement and restoration cycles.

Control and Diversity

  • Spatial and Categorical Control: A "Control Map" allows users to specify the exact location and type of defect (e.g., crack vs. rust).
  • Adaptive Noise Insertion: Stochastic Gaussian noise is injected into the decoder to ensure visual diversity, preventing the model from generating repetitive patterns.
  • Domain Adaptation: To ensure synthetic data is useful for real-world classifiers, a Gradient Reversal Layer (GRL) is used. This forces the inspection network to learn features common to both real and synthetic images, preventing it from over-fitting to synthesis artifacts.

Experiments and Results

The authors evaluated Defect-GAN using the CODEBRIM dataset, which contains images of concrete bridge defects (cracks, spallation, efflorescence, exposed bars, and corrosion).

Synthesis Quality

Defect-GAN was compared against state-of-the-art methods like CycleGAN and StyleGAN v2. Using the Frechet Inception Distance (FID) metric, Defect-GAN achieved a score of 65.6, significantly outperforming the next best competitor, StarGAN + SPADE, which scored 103.0.

Inspection Performance

The primary goal was to improve real-world inspection accuracy. When augmenting training data with Defect-GAN samples:

  • ResNet34 accuracy improved from 70.25% to 75.48%.
  • DenseNet121 accuracy improved from 70.77% to 75.79%.

Key Insights

A critical finding is that background preservation is paramount. By treating defects as layers rather than re-generating the entire image, the model avoids the "uncanny valley" of synthetic textures. Furthermore, the model demonstrates impressive transferability, successfully applying learned defects to entirely different surfaces, such as metal pipes, despite being trained on concrete.

Conclusion

Defect-GAN provides a robust solution for synthesizing high-fidelity defect data. While highly effective, future work may address challenges in synthesizing defects on complex 3D geometries and reducing dependency on initial image-level annotations. Optimizing the generator for real-time data augmentation also remains a promising direction for future research.

Summary updated

BibTeX
@inproceedings{zhang2021defect,
  author = {Gongjie Zhang and Kaiwen Cui and Tzu-Yi Hung and Shijian Lu},
  title = {{Defect-GAN: High-Fidelity Defect Synthesis for Automated Defect Inspection}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2021},
  url = {https://openreview.net/forum?id=BgDQkgRlJlw},
}
Figure