TL;DR
This paper introduces PE-GAN, a novel generative adversarial network (GAN) designed to create high-quality synthetic defect images for steel plates. By employing a unique "production-and-elimination" process, PE-GAN effectively guides the GAN to generate realistic defect samples, significantly improving the performance of automated defect inspection systems.
What Problem Does This Paper Address?
Ensuring the quality of steel plates is crucial, and surface defect inspection plays a vital role. While deep learning (DL) methods have shown great promise in automating this process, they are heavily reliant on large, diverse datasets. Collecting sufficient defect samples in real-world industrial settings is challenging due to the rarity and randomness of defects, leading to insufficient training data and suboptimal inspection accuracy. Existing data augmentation techniques are often too simplistic, and while GANs can generate synthetic data, they often produce low-quality or unrealistic samples that can hinder, rather than help, model training. This paper aims to bridge this gap by proposing a method to generate high-quality, realistic defect samples that can effectively augment existing datasets and boost the performance of DL-based inspection models.
Methods
The core of the proposed method is the PE-GAN (Production-and-Elimination Generative Adversarial Network). It builds upon the DCGAN architecture but introduces a novel two-stage generation process inspired by diffusion models.
- Production Stage: The generator learns to create defects on defect-free background samples. This simulates the actual formation of defects on steel surfaces.
- Elimination Stage: The generator then learns to remove defects from existing defective samples, effectively learning to reconstruct the background.
By minimizing the differences between samples at both stages, the PE-GAN is guided to produce more realistic defect and background images. This "production-and-elimination" process helps overcome the GAN's tendency to learn simple patterns and provides a more controlled learning direction.
Additionally, the authors incorporate metric learning into the adversarial training. This aims to minimize the distribution differences between generated background samples and real background samples, indirectly improving the realism of generated defect samples. The discriminator in PE-GAN has two output layers: one for distinguishing real vs. fake samples (adversarial loss) and another for classifying the defect type (classification loss). A reconstruction loss is also used to ensure generated samples preserve the content of the original domain while incorporating target domain features.
Datasets
The experiments were conducted on the NEU-CLS-64 defect dataset. This dataset consists of images of steel plates with nine classes of defects: crazing (Cr), grooves and gouges (GG), inclusion (In), patches (Pa), pitted surface (PS), rolling dust (RD), rolled-in scale (RS), scratches (Sc), and spots (Sp). The dataset is known to be imbalanced.
For training, the authors randomly selected 200 samples per defect class (to address imbalance) and 1000 background samples. The defect samples were split into a 7:3 training/testing ratio, with all background samples added to the training set.
Key Insights and Ideas
- Production-and-Elimination as a Guided GAN: The core insight is that by simulating both the creation and removal of defects, the GAN can be more effectively guided to learn realistic defect patterns, moving beyond simple feature generation.
- Leveraging Abundant Background Data: The method effectively utilizes the readily available defect-free background samples to train the generation process, addressing the data scarcity issue for defects.
- Metric Learning for Realism: Incorporating metric learning helps align the distribution of generated background samples with real ones, which indirectly enhances the quality of generated defect samples.
- Two-Stage Process for Control: The two-stage approach provides a more nuanced control over the generation process compared to standard GANs, leading to improved sample quality.
Key Results
- Improved Sample Quality: The PE-GAN achieved the lowest average Frechet Inception Distance (FID) score across nine defect classes compared to other generative methods (DCGAN, LSGAN, CycleGAN, StarGAN), indicating superior generation quality and realism.
- Enhanced Inspection Accuracy: When trained with PE-GAN generated samples, inspection models (ResNet-18 and MobileNet) showed significant accuracy improvements. For ResNet-18, accuracy increased from 93.24% (no generated samples) to 99.79% with PE-GAN samples. For MobileNet, it increased from 91.26% to 99.75%.
- Outperforming Other GANs: The PE-GAN consistently outperformed other image-to-image translation GANs in terms of FID scores and inspection accuracy, demonstrating its effectiveness.
Limitations and Future Work
- Specific Defect Types: The paper notes that for certain defect types with very small features (e.g., Cr, RS), generation results were poorer, making them easily confused with surface texture. This suggests that highly subtle or small defects remain a challenge.
- Future Work: The authors suggest that the PE-GAN can guide the learning direction for generating higher quality and more realistic defect samples, implying that further refinement of the generation process could lead to even better results. The success in improving inspection accuracy also points towards further exploration of how generated data can be optimally integrated into training pipelines.
Summary updated