IoT-Enabled Few-Shot Image Generation for Power Scene Defect Detection Based on Self-Attention and Global-Local Fusion

Yi Chen, Yunfeng Yan, Xianbo Wang, Yi ZhengSensors 2023 · OpenReview

TL;DR

This paper introduces MVSA-GAN, a novel few-shot image generation framework for detecting defects in power equipment. By leveraging self-attention and a global-local feature fusion module, MVSA-GAN generates realistic and diverse defect images from limited data, significantly improving downstream defect detection tasks.

What Problem Does This Paper Address?

Detecting defects in power equipment is crucial for safety and reliability. However, current machine learning approaches struggle due to several challenges:

  • Data Scarcity: Acquiring large, diverse datasets of power scene defects is difficult due to privacy concerns, security issues, and the imbalanced nature of defect categories.
  • Limited Learning Ability: Existing models require extensive data to achieve high detection accuracy and generalization.
  • Complex Scenarios: Power equipment operates in dynamic and complex environments, making it hard to capture all relevant defect variations.

The paper aims to address these issues by enabling effective defect detection even with limited training data, a scenario known as few-shot learning.

Methods

The core of the proposed solution is the MVSA-GAN (Multi-View Self-Attention Generative Adversarial Network). It comprises several key components:

  • IoT Integration: The system utilizes IoT devices to collect real-time data from power equipment, providing a foundation for generating synthetic defect data.
  • Self-Attention Encoder (SAEncoder): This module employs a Transformer-based structure to capture contextual information within input images. By focusing on relevant features across different image parts, it enhances the authenticity and coherence of generated images, especially for complex scenes.
  • Global-Local Fusion Module (GLFM): This module is designed to fuse both global and local features from input images. This allows the model to capture the overall scene context (global) while also preserving fine-grained details of defects (local), leading to more realistic and diverse generated images.
  • Generative Adversarial Network (GAN) Framework: MVSA-GAN uses a GAN architecture, consisting of a generator (MVSA-GAN) and a discriminator, to learn to produce synthetic images that are indistinguishable from real defect images.

Datasets

The study utilizes a large dataset of 59,169 power scene defect images covering 12 categories, including blurred dials, broken dials, insulator ruptures, and metal corrosion.

For training the MVSA-GAN, a subset of 4800 images was used, randomly selected from each category. During training, 10 categories were treated as "visible" (available for learning), while the remaining 2 categories were "invisible" (used to test few-shot generation capabilities).

The generated data was then used to train downstream classification networks (ResNet50 and VGG16) on datasets with and without synthetic data for evaluation.

Key Insights and Ideas

  • Self-Attention for Context: The SAEncoder's use of self-attention is crucial for understanding the broader context of an image, which is vital for generating realistic defect images in complex power environments where defects might be subtle or obscured.
  • Global-Local Fusion for Realism: Combining global and local features is key to generating images that are both structurally sound and rich in detail. This approach helps overcome the limitations of methods that focus solely on local features, which can lead to artifacts or loss of overall scene coherence.
  • IoT as a Data Enabler: Integrating IoT devices provides a practical pathway to collect real-time operational data, which can then be leveraged to generate more relevant and diverse synthetic defect data, addressing the scarcity problem.
  • Few-Shot Learning for Practicality: The paper demonstrates that even with very limited examples of a specific defect, the proposed method can generate high-quality synthetic data, making it highly applicable to real-world scenarios where data collection is challenging.

Key Results

  • Quantitative Performance: MVSA-GAN achieved state-of-the-art results on key image quality metrics, scoring 67.87 for FID and 0.179 for LPIPS. These scores indicate superior realism and similarity to real images compared to other methods like FIGR and DAWSON.
  • Downstream Task Improvement: When using generated data to train classification networks (ResNet50 and VGG16), the F1 scores significantly improved. For example, ResNet50 saw an increase from 89.0% to 92.8% for "Meter Breakage" when trained with generated data.
  • Qualitative Results: Visualizations show that MVSA-GAN generates defect images that are realistic, diverse, and accurately depict defect locations and backgrounds, outperforming other methods in terms of visual quality.

Limitations and Future Work

  • Data Availability: While the paper uses a substantial dataset, the authors note that the generated data is currently private due to ongoing research. Future work will involve making this data public.
  • Computational Cost: The use of Transformer-based architectures and GANs can be computationally intensive. While not explicitly detailed as a limitation, optimizing for efficiency could be a future direction.
  • Generalization to Unseen Defects: The paper focuses on few-shot generation for existing defect categories. Extending this to generate entirely novel defect types could be a future research avenue.

Summary updated

Abstract

Defect detection in power scenarios is a critical task that plays a significant role in ensuring the safety, reliability, and efficiency of power systems. The existing technology requires enhancement in its learning ability from large volumes of data to achieve ideal detection effect results. Power scene data involve privacy and security issues, and there is an imbalance in the number of samples across different defect categories, all of which will affect the performance of defect detection models. With the emergence of the Internet of Things (IoT), the integration of IoT with machine learning offers a new direction for defect detection in power equipment. Meanwhile, a generative adversarial network based on multi-view fusion and self-attention is proposed for few-shot image generation, named MVSA-GAN. The IoT devices capture real-time data from the power scene, which are then used to train the MVSA-GAN model, enabling it to generate realistic and diverse defect data. The designed self-attention encoder focuses on the relevant features of different parts of the image to capture the contextual information of the input image and improve the authenticity and coherence of the image. A multi-view feature fusion module is proposed to capture the complex structure and texture of the power scene through the selective fusion of global and local features, and improve the authenticity and diversity of generated images. Experiments show that the few-shot image generation method proposed in this paper can generate real and diverse defect data for power scene defects. The proposed method achieved FID and LPIPS scores of 67.87 and 0.179, surpassing SOTA methods, such as FIGR and DAWSON.
BibTeX
@inproceedings{chen2023iot,
  author = {Yi Chen and Yunfeng Yan and Xianbo Wang and Yi Zheng},
  title = {{IoT-Enabled Few-Shot Image Generation for Power Scene Defect Detection Based on Self-Attention and Global-Local Fusion}},
  booktitle = {Sensors},
  year = {2023},
  url = {https://openreview.net/forum?id=6jOpei28ay},
}
Figure