PDeT: A Progressive Deformable Transformer for Photovoltaic Panel Defect Segmentation

Peng Zhou, Hong Fang, Gaochang WuSensors 2024 · DOI

TL;DR

This paper introduces PDeT, a novel Progressive Deformable Transformer for precise defect segmentation in photovoltaic (PV) panels. By adaptively sampling features and progressively fusing semantic information, PDeT achieves state-of-the-art performance, crucial for ensuring the efficiency and safety of solar energy systems.

What Problem Does This Paper Address?

Photovoltaic (PV) panels are essential for clean energy, but defects can significantly reduce their power generation efficiency and even cause safety hazards like hotspots. Accurate, pixel-level defect segmentation is vital for quality control and maintenance. Existing deep learning methods, particularly those based on Convolutional Neural Networks (CNNs), struggle with precisely localizing elongated defects and can miss fine details due to their fixed feature sampling. While Vision Transformers (ViTs) offer better long-range dependency capture, their dense attention mechanism is computationally expensive. Furthermore, effectively fusing coarse semantic information with fine-grained details in the decoder remains a challenge, often leading to diluted information.

Methods

The proposed PDeT model tackles these challenges with two key innovations:

  • Progressive Deformable Transformer (PDeT) Architecture: PDeT employs an encoder-decoder structure. The encoder utilizes a Deformable Transformer Block (DTB). Unlike standard self-attention, the DTB incorporates a deformable sampling mechanism. This mechanism learns spatial offsets for sampling key and value features, allowing the model to dynamically adjust its receptive field and focus on irregular or complex defect shapes. This is achieved by generating reference points and then learning offsets for these points, enabling more precise feature extraction.
  • Semantic Aggregation Block (SAB): In the decoder, the SAB is introduced to effectively fuse multi-scale features from different encoder stages. It aligns channel dimensions, upsamples higher-level semantic features to match lower-level ones, and then fuses them. This process is repeated with multi-scale pooling and residual connections to preserve both coarse semantic context and fine-grained details, preventing information dilution and improving segmentation accuracy.

Datasets

The paper uses two primary datasets:

  • Solar Cell Defect Dataset: A custom dataset of 1024 PV panel images, split into training and validation sets at a 2:1 ratio. Images are 640x640 resolution and segmented into "defect" and "background" categories. Random cropping was used for data augmentation.
  • MVTec-AD Dataset: A public dataset for anomaly detection, used to validate PDeT's generalizability. Four specific scenes (Wood, Metal Nut, Tile, and Hazelnut) were selected, representing diverse complex backgrounds and defect types. Data was stratified sampled and split into training and testing sets at a 2:1 ratio.

Key Insights and Ideas

  • Deformable Attention for Irregular Defects: The core idea is that defects, especially cracks, are often irregular and elongated. Standard fixed sampling in Transformers (and CNNs) is suboptimal. By learning where to sample features (via offsets), the model can better adapt to these shapes.
  • Progressive Feature Fusion is Crucial: Simply concatenating features isn't enough. The SAB's approach of aligning, upsampling, and fusing features across scales, while preserving original information via residual connections, is key to retaining both high-level understanding and low-level detail.
  • Transformer's Adaptability: The paper highlights how the Transformer architecture, enhanced with deformable attention, can achieve superior performance in complex scenarios where traditional methods falter, particularly with fine-grained details like crack edges.

Key Results

  • PV Defect Segmentation: PDeT achieved a mean Intersection over Union (mIoU) of 88.41% on the solar cell defect dataset, outperforming other state-of-the-art methods.
  • MVTec-AD Performance: PDeT demonstrated strong adaptability and stability across various industrial scenarios on the MVTec-AD dataset, achieving competitive mIoU scores (e.g., 94.34% for Hazelnut, 95.32% for Metal Nut).
  • Ablation Studies: Replacing standard Transformer blocks with Deformable Transformer Blocks (DTBs) progressively improved performance, with full integration yielding a significant gain. The Semantic Aggregation Block (SAB) also contributed to performance improvements, with sequential integration showing increasing benefits.
  • Comparison with Other Decoders: PDeT's proposed decoder head outperformed FPN and UPerNet heads, achieving a higher defect class IoU (76.85%) and overall mIoU (88.41%).

Limitations and Future Work

  • Computational Cost: The paper acknowledges that while PDeT achieves high accuracy, it has a relatively large number of parameters, leading to increased computational overhead.
  • Future Work: The authors plan to explore pruning and quantization methods suitable for Transformer architectures to optimize parameter count and inference efficiency.

Summary updated

Abstract

Defects in photovoltaic (PV) panels can significantly reduce the power generation efficiency of the system and may cause localized overheating due to uneven current distribution. Therefore, adopting precise pixel-level defect detection, i.e., defect segmentation, technology is essential to ensuring stable operation. However, for effective defect segmentation, the feature extractor must adaptively determine the appropriate scale or receptive field for accurate defect localization, while the decoder must seamlessly fuse coarse-level semantics with fine-grained features to enhance high-level representations. In this paper, we propose a Progressive Deformable Transformer (PDeT) for defect segmentation in PV cells. This approach effectively learns spatial sampling offsets and refines features progressively through coarse-level semantic attention. Specifically, the network adaptively captures spatial offset positions and computes self-attention, expanding the model's receptive field and enabling feature extraction across objects of various shapes. Furthermore, we introduce a semantic aggregation module to refine semantic information, converting the fused feature map into a scale space and balancing contextual information. Extensive experiments demonstrate the effectiveness of our method, achieving an mIoU of 88.41% on our solar cell dataset, outperforming other methods. Additionally, to validate the PDeT's applicability across different domains, we trained and tested it on the MVTec-AD dataset. The experimental results demonstrate that the PDeT exhibits excellent recognition performance in various other scenarios as well.
BibTeX
@inproceedings{zhou2024pdet,
  author = {Peng Zhou and Hong Fang and Gaochang Wu},
  title = {{PDeT: A Progressive Deformable Transformer for Photovoltaic Panel Defect Segmentation}},
  booktitle = {Sensors},
  year = {2024},
  doi = {10.3390/s24216908},
}
Figure