Large Span Sizes and Irregular Shapes Target Detection Methods Using Variable Convolution-Improved YOLOv8

Yan Gao, Wei Liu, Hsiang‐Chen Chui, Xiaoming ChenSensors 2024 · DOI

TL;DR

This paper tackles the challenge of accurately detecting large, irregularly shaped objects in industrial settings, specifically scrap steel. They achieve this by enhancing the YOLOv8 architecture with deformable convolutions for better shape adaptation, a parameter-free attention mechanism (Sim-AM) to focus on crucial features, and a simplified focal modulation network to speed up processing. The result is a more robust and efficient object detection system for complex industrial scenes.

What Problem Does This Paper Address?

Detecting objects in industrial environments, particularly scrap steel, presents several significant challenges. Scrap steel often comes in a wide variety of shapes and sizes, with many objects being large and irregularly formed. This makes it difficult for standard object detection models, which often struggle with irregular shapes and varying scales. Furthermore, issues like low-resolution labeling, dense object distribution, and potential for overlap can lead to low accuracy, missed detections (leakage), and misclassifications. Existing deep learning models, while powerful, often require extensive datasets and can be computationally expensive, making real-time industrial application a hurdle. This paper aims to bridge this gap by improving the accuracy and efficiency of object detection for these challenging industrial targets.

Methods

The core of this work is the modification of the YOLOv8 object detection framework. The authors introduce three key enhancements:

  • Deformable Convolutional Network (DCN) Integration: The standard convolutional kernels in YOLOv8's backbone network are replaced with deformable convolutional blocks. This allows the convolution kernel to adapt its shape and size based on the input data, enabling it to better capture the irregular contours of scrap steel and improve the model's receptive field. This directly addresses the issue of detecting objects with non-standard shapes.
  • Sim-AM (Simple Parameter-Free Attention Mechanism): To combat the problem of small targets being obscured by background noise and to enhance the focus on relevant features, the Sim-AM module is integrated. This mechanism directly infers 3D weights from neurons, optimizing both spatial and channel attention without adding extra parameters. This makes the model more efficient while improving its ability to discern important features.
  • Focal Modulation Network Replacement: The Spatial Pyramid Pooling (SPPF) module in YOLOv8's backbone is replaced with a focal modulation network. This new module simplifies the computational process by encoding visual contexts at different granularities and selectively aggregating them. This leads to a more streamlined model architecture and faster processing speeds.

Datasets

The primary dataset used for training and evaluation in this study is a custom scrap steel dataset. This dataset was collected from the Ling-yuan Iron and Steel Group and comprises 16,071 photos and 3 video recordings of real-time scrap steel unloading. The dataset is categorized into five types: light scrap, medium scrap, heavy scrap, oily scrap, and confined scrap. A portion (20%) of this dataset was randomly allocated for testing.

Additionally, the authors performed a generalization experiment using the publicly available PASCAL VOC dataset to validate the stability and broader applicability of their improved model.

Key Insights and Ideas

  • Adaptive Receptive Fields are Crucial for Irregular Shapes: The success of the deformable convolution module highlights how critical it is for object detectors to adapt their "view" to the object's actual shape, rather than relying on fixed kernel shapes. This is especially true for industrial materials like scrap steel.
  • Parameter-Free Attention for Efficiency: The Sim-AM module's ability to improve attention without increasing model complexity or parameters is a significant win for real-time applications. It shows that sophisticated feature enhancement doesn't always require a heavy computational footprint.
  • Simplification Can Boost Speed: Replacing the SPPF with a focal modulation network demonstrates that architectural choices can drastically impact inference speed. Streamlining the feature aggregation process is key to achieving real-time performance.
  • Oily Scrap Remains a Challenge: Despite improvements, the oily scrap category, due to its susceptibility to environmental interference (rust, shadows) and potentially smaller sample size, still presents a notable challenge, indicating areas for future research.

Key Results

The improved YOLOv8 model demonstrated significant performance gains:

  • Accuracy Improvements:
    • Average Precision (AP) increased by 2.1%.
  • Mean Average Precision (mAP) increased by 0.8%.
  • Specific class improvements were noted, with light scrap up by 1%, medium scrap by 1.2%, heavy scrap by 0.4%, and confined scrap by 1.5%.
  • Efficiency Gains:
    • Frames Per Second (FPS) was reduced by 5.4, indicating a faster inference speed.
  • The paper states the improved model achieves a detection speed of 0.003s per image, meeting real-time industrial requirements.
  • Generalization: On the PASCAL VOC dataset, the improved model showed a substantial 15.5% increase in mAP compared to the baseline YOLOv8, demonstrating its robustness across different object types and datasets.

Limitations and Future Work

  • Oily Scrap Detection: The paper acknowledges that the oily scrap category, due to its unique challenges (environmental interference, potential data imbalance), remains the most difficult to detect accurately. The recall for this category is still the lowest.
  • Future Work Suggestions:
    • The authors suggest exploring a classifier fusion approach for oily scrap. This would involve using a specialized classifier for oily scrap and then integrating its results with those from classifiers for easier-to-detect scrap types.
  • Further research could focus on addressing the specific challenges of oil scrap detection, potentially through more targeted data augmentation or specialized network modules.

Summary updated

Abstract

In this work, an object detection method using variable convolution-improved YOLOv8 is proposed to solve the problem of low accuracy and low efficiency in detecting spanning and irregularly shaped samples. Aiming at the problems of the irregular shape of a target, the low resolution of labeling frames, dense distribution, and the ease of overlap, a deformable convolution module is added to the original backbone network. This allows the model to deal flexibly with the problem of the insufficient perceptual field of the target corresponding to the detection point, and the situations of leakage and misdetection can be effectively improved. In order to solve the issue that small target detection is susceptible to image background and noise interference, the Sim-AM (simple parameter-free attention mechanism) module is added to the backbone network of YOLOv8, which enhances the attention to the underlying features and, thus, improves the detection accuracy of the model. More importantly, the Sim-AM module does not need to add parameters to the original network, which reduces the computation of the model. To address the problem of complex model structures that can lead to slower detection, the spatial pyramid pooling of the backbone network is replaced with focal modulation networks, which greatly simplifies the computation process. The experimental validation was carried out on the scrap steel dataset containing a large number of targets of multiple shapes and sizes. The results showed that the improved YOLOv8 network model improves the AP (average precision) by 2.1%, the mAP (mean average precision value) by 0.8%, and reduces the FPS (frames per second) by 5.4, which meets the performance requirements of real-time industrial inspection.
BibTeX
@inproceedings{gao2024large,
  author = {Yan Gao and Wei Liu and Hsiang‐Chen Chui and Xiaoming Chen},
  title = {{Large Span Sizes and Irregular Shapes Target Detection Methods Using Variable Convolution-Improved YOLOv8}},
  booktitle = {Sensors},
  year = {2024},
  doi = {10.3390/s24082560},
}
Figure