Mixed Receptive Fields Augmented YOLO with Multi-Path Spatial Pyramid Pooling for Steel Surface Defect Detection

Kewen Xia, Zhongliang Lv, Chuande Zhou, Guojun Gu, Zhiqiang Zhao, Kang Liu, Zelun LiSensors 2023 · DOI

TL;DR

This paper introduces an enhanced YOLOv5s model for detecting defects on steel surfaces, tackling challenges like complex textures and varying defect scales. By incorporating a novel "re-parameterized large kernel C3 module" and a "multi-path spatial pyramid pooling" structure, the model significantly improves detection accuracy and efficiency, achieving a 76.8% mAP on the NEU-DET dataset.

What Problem Does This Paper Address?

Detecting defects on steel surfaces is crucial for quality control in manufacturing. However, this task is complicated by several factors:

  • Texture Interference: Steel surfaces often have intricate textures that can easily be mistaken for defects or mask actual defects.
  • Scale Variation: Defects can appear at vastly different sizes, from tiny cracks to larger patches, making it difficult for a single model to capture them all effectively.
  • Subtle Defects: Some defects are very subtle, with low contrast and indistinct boundaries, further challenging detection algorithms.
  • Efficiency and Lightweight Deployment: Industrial applications often require fast and efficient models that can run on embedded systems, posing a challenge for complex deep learning architectures.

Traditional image processing and early machine learning methods struggle with these issues, while existing deep learning approaches, though better, still face limitations in robustness and accuracy under these challenging conditions.

Methods

The researchers propose an improved YOLOv5s architecture with several key innovations:

  • Re-parameterized Large Kernel C3 (RepLK-C3) Module: This novel module replaces the standard C3 module in YOLOv5. It utilizes a re-parameterized large kernel convolution, which effectively expands the receptive field of the network. This allows the model to capture more global context and better distinguish between actual defect features and background textures. The re-parameterization technique helps to reduce computational cost while achieving the benefits of a larger kernel.
  • Multi-Path Spatial Pyramid Pooling (RepLK-SPP) Neck: The standard spatial pyramid pooling fast (SPPF) module in YOLOv5 is enhanced and integrated into a multi-path structure within the neck of the network. This module uses multiple large kernel convolutions to extract features at different receptive fields. By processing feature maps from different scales in parallel branches, it aims to capture more precise positional information and adapt to varying defect sizes.
  • Mixed Kernel Size Training Strategy: The paper proposes a training strategy where different kernel sizes are applied to feature maps at different scales. This allows the model to dynamically adapt its receptive field to the scale of the features it's processing, further improving its ability to handle scale variations.
  • Additional P6 Layer: An extra P6 layer is added to the backbone network to accommodate the detection of smaller objects and improve multi-scale defect detection.

Datasets

The experiments were conducted on the NEU-DET dataset. This dataset comprises 1800 grayscale images of hot-rolled steel surface defects, with 300 images for each of the six defect types:

  • Crazing (Cr)
  • Inclusion (In)
  • Patches (Pa)
  • Pitted surface (Ps)
  • Rolled-in scale (RS)
  • Scratches (Sc)

The dataset was split into training (1440 images), validation (180 images), and test (180 images) sets.

Key Insights and Ideas

  • Large Kernels for Texture Discrimination: The study highlights that larger convolutional kernels, when implemented efficiently (e.g., through re-parameterization), are crucial for distinguishing subtle defect features from complex background textures. They allow the model to "see" more context and focus on shape rather than just local patterns.
  • Multi-Path SPP for Scale Adaptation: Simply stacking pooling layers isn't enough for multi-scale feature fusion. A multi-path approach that explicitly extracts features at different receptive fields and fuses them is more effective for capturing the diverse scales of defects.
  • Adaptive Receptive Fields via Training: The idea of dynamically adjusting the receptive field based on feature map scale during training is a clever way to make the model more robust to scale variations without overly complicating the architecture.
  • Trade-offs in Kernel Size: The experiments show that while larger kernels generally improve accuracy, there's an optimal range (around 25x25 in their tests). Exceeding this can lead to diminishing returns or even performance degradation, especially for smaller feature maps. This justifies the mixed-kernel approach.

Key Results

The proposed improved YOLOv5s model demonstrated significant improvements:

  • Mean Average Precision ([email protected]): Achieved 76.8%, an increase of 8.6% over the original YOLOv5s and 3.7% over YOLOv8.
  • Specific Defect Improvements:
    • Crazing and rolled-in scale (weak texture, dense): +14.4% and +11.1% accuracy, respectively.
  • Inclusion and scratches (prominent scale changes, significant shape): +10.5% and +6.6% accuracy, respectively.
  • Efficiency: The model maintains a lightweight footprint with a volume of 28.9 MB, significantly smaller than comparative models like RetinaNet-SW (282.0 MB) and Faster R-CNN (41.2 MB).
  • Speed: Achieved a detection speed of 67.1 FPS, meeting real-time detection requirements.

Limitations and Future Work

  • Limitations: The paper notes that while the model improves detection accuracy, the model volume increased slightly compared to the original YOLOv5s. The authors also mention that their current work focuses on mixed receptive fields for steel surfaces and doesn't delve into other applications.
  • Future Work: The authors suggest exploring the application of their mixed receptive fields model in other related fields.

Summary updated

Abstract

Aiming at the problems of low detection efficiency and poor detection accuracy caused by texture feature interference and dramatic changes in the scale of defect on steel surfaces, an improved YOLOv5s model is proposed. In this study, we propose a novel re-parameterized large kernel C3 module, which enables the model to obtain a larger effective receptive field and improve the ability of feature extraction under complex texture interference. Moreover, we construct a feature fusion structure with a multi-path spatial pyramid pooling module to adapt to the scale variation of steel surface defects. Finally, we propose a training strategy that applies different kernel sizes for feature maps of different scales so that the receptive field of the model can adapt to the scale changes of the feature maps to the greatest extent. The experiment on the NEU-DET dataset shows that our model improved the detection accuracy of crazing and rolled in-scale, which contain a large number of weak texture features and are densely distributed by 14.4% and 11.1%, respectively. Additionally, the detection accuracy of inclusion and scratched defects with prominent scale changes and significant shape features was improved by 10.5% and 6.6%, respectively. Meanwhile, the mean average precision value reaches 76.8%, compared with the YOLOv5s and YOLOv8s, which increased by 8.6% and 3.7%, respectively.
BibTeX
@inproceedings{xia2023mixed,
  author = {Kewen Xia and Zhongliang Lv and Chuande Zhou and Guojun Gu and Zhiqiang Zhao and Kang Liu and Zelun Li},
  title = {{Mixed Receptive Fields Augmented YOLO with Multi-Path Spatial Pyramid Pooling for Steel Surface Defect Detection}},
  booktitle = {Sensors},
  year = {2023},
  doi = {10.3390/s23115114},
}
Figure