TL;DR
This paper introduces LarGAN, a novel generative adversarial network (GAN) designed to address the critical issue of insufficient data for rare surface defect detection in industrial settings, particularly for continuous casting slabs. LarGAN synthesizes high-quality, diverse defect images from single examples using a progressive training approach and a unique "Label Auto-Rescaling" strategy, significantly improving the performance of downstream object detection models.
What Problem Does This Paper Address?
Industrial production, especially in areas like continuous casting slabs, often suffers from a scarcity of defect data. This is primarily due to the fast-paced nature of these environments, making it difficult and expensive to collect a comprehensive dataset of rare surface defects like scratches, cracks, and slag marks. This data limitation directly hinders the accuracy and reliability of automated defect detection systems, which are increasingly relying on deep learning. Traditional manual inspection is inefficient and labor-intensive. While deep learning, particularly Convolutional Neural Networks (CNNs), shows promise, it requires substantial amounts of data. Existing data augmentation techniques, especially those based on generative models, often struggle with generating realistic and diverse defect images from limited samples, leading to issues like chaotic structures or a lack of fine-grained detail.
Methods
LarGAN employs a progressive Generative Adversarial Network (GAN) framework. This means the model starts by generating low-resolution images and gradually increases the network's depth and resolution in subsequent stages. This progressive approach ensures a smoother and more stable generation process.
The core innovation is the Label Auto-Rescaling strategy. Instead of relying solely on the input image, LarGAN uses the defect's label information (bounding box) to guide the generation. This strategy progressively expands the defect area within the label, creating a "pseudo-scale image pyramid" with varying resolutions and foreground-to-background ratios. This allows the generator to first focus on learning the essential defect features (foreground) and then progressively incorporate background details, leading to more controlled and semantically accurate defect synthesis.
The model also incorporates a rescaling rate and learning rate adjustment mechanism to control the diversity of generated defects. This involves neither freezing nor completely retraining weights from previous stages, striking a balance to ensure diversity without losing learned defect information.
Datasets
The primary dataset used is the "Continuous Casting Slab Defects" dataset. This dataset comprises 2400 images of varying sizes, featuring six categories of rare defects: longitudinal cracks (Lc), scratches (Sc), water slag marks (WSM), welding slag (Ws), slag skin (Ss), and cutting openings (Co). Each category has 400 images.
For further validation of generalization capabilities, LarGAN was also tested on a steel pipe surface defect dataset. This dataset contained four defect categories (Warp, External fold, Wrinkle, Scratch) with 554 defective and 673 non-defective samples.
Key Insights and Ideas
- Label-guided generation is crucial for rare defects: Simply generating images similar to an input image isn't enough for industrial defects. Using label information to guide the generation process allows for better control over defect characteristics and semantics.
- Progressive generation aids stability and quality: Building up image resolution and network complexity gradually, rather than attempting high-resolution generation from the start, leads to more stable training and higher-quality outputs.
- Balancing diversity and realism is key: The method's approach to weight training (not freezing, not retraining completely) and the label auto-rescaling strategy are designed to produce diverse defects that are still realistic and representative of actual flaws.
- Data augmentation with LarGAN directly boosts downstream model performance: The generated images aren't just visually appealing; they demonstrably improve the accuracy of object detection models like YOLOv8, especially in low-data scenarios.
Key Results
- Image Quality: LarGAN outperformed other single-image generation models (SinGAN, ConSinGAN) in terms of image quality, as measured by LPIPS and SSIM metrics. For instance, LarGAN achieved a lower LPIPS score (indicating higher perceptual similarity) across various defect types compared to SinGAN and ConSinGAN.
- Dataset Diversity: LarGAN achieved a significantly lower FID score (64.3) compared to ConSinGAN (73.8) and SinGAN (104.1), indicating that its generated images are more similar to the real dataset distribution.
- Object Detection Performance: When used to augment a dataset for YOLOv8 training, LarGAN significantly improved detection accuracy. In a scenario with only 10% of the original data, LarGAN-generated images led to a 6.2 AP increase in model accuracy.
- Generalization: LarGAN successfully generated defect images on a different dataset (steel pipe surface defects), demonstrating its ability to adapt to different industrial contexts, although some edge artifacts were observed.
Limitations and Future Work
- Mode Collapse: The paper acknowledges encountering "mode collapse," where the generator produces overly simplistic outputs (e.g., dark or yellow images). This was attributed to low information entropy in dark or low-contrast input images. The authors suggest increasing image contrast and brightness during preprocessing to mitigate this.
- Complex Backgrounds: LarGAN's generalization ability is noted as limited in complex backgrounds. When applied to the steel pipe dataset, variations in pipe diameter and edge regions sometimes introduced artifacts. Future work could involve filtering irrelevant edge content or improving background complexity handling.
- Future Work:
- Broader Industrial Applications: Expanding LarGAN's use beyond casting slabs to other industries like electronics and automotive manufacturing.
- Multi-Defect Generation: Developing methods to generate images with multiple defects or joint distributions of various defects, as the current method focuses on single defect types.
- Advanced Generative Models: Exploring newer models like Diffusion Models, Vision Transformers, or Multimodal Large Models to further enhance generation quality and diversity.
Summary updated