Temporal-Quality Ensemble Technique for Handling Image Blur in Packaging Defect Inspection

Guk-Jin Son, Heechul Jung, Young-Duk Kim2024 · DOI

TL;DR

This paper introduces a novel "Temporal-Quality Ensemble" (TQE) technique to improve packaging defect detection, especially when dealing with blurry images. By intelligently weighting images based on both their temporal order and visual quality, TQE significantly boosts accuracy and reliability compared to standard methods, even when only a few blurry images are present.

What Problem Does This Paper Address?

Automated visual inspection systems are crucial for quality control in manufacturing, particularly for packaging. While deep learning models have shown great promise in identifying surface defects, they often struggle with "low-quality" images. In industrial settings, these low-quality images are frequently caused by motion blur (due to conveyor movement) or focus blur (out-of-focus issues). These blur artifacts can obscure critical details, like ingredient lists, and significantly degrade the performance of standard Convolutional Neural Networks (CNNs). Existing methods often assume high-quality images or simply discard blurry ones, leading to missed defects. This paper tackles the challenge of robust packaging defect inspection in environments where a mix of high-quality and blurry images is common.

Methods

The core of this paper is the proposed Temporal-Quality Ensemble (TQE) technique, designed to be used during the inference (testing) phase, not training. TQE combines two key weighting strategies:

  • Temporal Weighting: This assigns importance to images based on their temporal proximity to the current "observed" image. Images captured more recently receive higher weights, acknowledging that consecutive frames in a video stream often share similar features. This is formulated using an exponential decay function based on the time difference.
  • Quality Weighting: This prioritizes images with better visual clarity. The paper uses the Laplacian variance as a measure of image sharpness. Higher Laplacian variance indicates sharper features and thus higher quality. This weight ensures that clearer images contribute more to the final decision.

TQE calculates a combined weight by averaging the temporal and quality weights for each input image. These combined weights are then used to ensemble the predictions from a pre-trained CNN model, effectively balancing the influence of both temporal information and image clarity. The authors also propose a binary search algorithm to help determine an optimal parameter for the temporal weighting.

Datasets

The authors collected a new dataset specifically for this research, as existing public datasets often lack a sufficient mix of high-quality and low-quality (blurred) packaging images.

  • Custom Dataset: This dataset consists of 9000 packaging images, resized to 256x256 pixels. It includes three defect classes: non-defect, edge defect, and surface defect, with 3000 images per class.
  • High-Quality Images: These were captured under controlled lighting and stable conditions.
  • Low-Quality Images: These were intentionally created to simulate motion blur (using a conveyor belt at 15 cm/s) and focus blur (by randomly changing camera focus). These low-quality images were selected based on having Laplacian variance values of 50% or less compared to high-quality images. The low-quality subset includes 1500 images for motion blur and 1500 for focus blur, with 500 images per class for each blur type.
  • Training/Validation/Test Split: The dataset was split into training, validation, and test sets, with 3000 images in each.

Key Insights and Ideas

  • Blur is a Major Hurdle: The paper strongly emphasizes that motion and focus blur are significant challenges for current deep learning models in industrial inspection, leading to substantial performance drops.
  • Ensembling is Key, but Quality Matters: While ensemble methods generally outperform single models, simply averaging predictions (like in AVE) can be detrimental when low-quality images are present, as they can unduly influence the outcome.
  • Temporal Context is Valuable: Even blurry images, when considered in their temporal sequence, can provide useful information that might be missed by static analysis.
  • Quality Prioritization is Crucial: Explicitly weighting images by their sharpness is essential for robust defect detection in mixed-quality scenarios.
  • Low-Quality Images Can Offer Unique Features: Surprisingly, the authors found that in some scenarios, ensembles including low-quality images performed better than those with only high-quality images. This suggests that blur might sometimes reveal or highlight certain defect features that are less apparent in sharp images.

Key Results

  • Single CNN Performance: When trained on high-quality images, CNN models showed high accuracy (over 94%) on high-quality test images but dropped significantly (10-50%) on low-quality images.
  • TQE vs. Single CNNs: TQE achieved an F1-score approximately 17.64% to 22.41% higher than single CNN models when at least one low-quality image was included in the input.
  • TQE vs. AVE: TQE outperformed the average voting ensemble (AVE) by about 1.86% to 2.06% in F1-score, demonstrating its ability to handle mixed-quality images more effectively.
  • Robustness: TQE showed consistent performance across various combinations of high- and low-quality images, outperforming AVE, which suffered more in scenarios dominated by low-quality images.
  • Speed vs. Accuracy Trade-off: While TQE significantly improved accuracy, it did lead to a decrease in processing speed (FPS) compared to AVE, a common trade-off in ensemble methods.

Limitations and Future Work

  • Limitations:
    • The study focused on motion and focus blur. Other types of image degradation (e.g., noise, low illumination) were not explicitly addressed.
  • The FPS decrease with TQE is a practical limitation for real-time applications.
  • Future Work:
    • Expand the analysis to include more diverse defect patterns.
  • Investigate other types of environmental factors that can degrade image quality, such as pollution or damage.
  • Explore methods to mitigate the FPS reduction while retaining TQE's accuracy benefits.

Summary updated

BibTeX
@misc{son2024temporal,
  author = {Guk-Jin Son and Heechul Jung and Young-Duk Kim},
  title = {{Temporal-Quality Ensemble Technique for Handling Image Blur in Packaging Defect Inspection}},
  year = {2024},
  doi = {10.3390/s24144438},
}

Loading related content...

Figure