TL;DR
This paper introduces a novel deep learning approach for classifying printing defects, specifically addressing the challenge of imbalanced datasets where some defect types are much rarer than others. By combining a custom deep convolutional neural network (DCNN) for feature extraction with various over-sampling techniques applied to these features, the authors achieve significant improvements in classification accuracy, particularly for minority defect classes.
What Problem Does This Paper Address?
Automated defect detection and classification are crucial in modern manufacturing, including the printing industry, to ensure product quality and minimize waste. While machine vision systems can often detect defects, accurately classifying their types remains a challenge. This is particularly problematic in printing due to:
- Imbalanced Datasets: Some printing defects occur very rarely, leading to datasets with a severe imbalance in the number of samples per defect class. Traditional classifiers struggle with such imbalances, often performing poorly on minority classes.
- Feature Extraction: Existing methods rely on hand-crafted features, which may not capture the most discriminative characteristics of defects directly from raw images.
- Practicality: Collecting large, representative datasets from real-world printing processes is difficult and costly.
This paper aims to bridge this gap by developing a robust deep learning method that can effectively classify printing defects even when sample distributions are highly uneven.
Methods
The proposed method involves a three-stage framework:
- Deep Model Training: The core of the approach is a custom-designed Deep Convolutional Neural Network (DCNN). This network is built to extract deep, discriminative features directly from printed defect images. The DCNN architecture includes convolutional blocks with convolution, batch normalization (BN), and ReLU activation, followed by max-pooling layers. The authors also explore using pre-trained deep models (transfer learning) as feature extractors for comparison.
- Over-sampling of Deep Features: To address the imbalanced dataset problem, the authors investigate seven different over-sampling techniques. Crucially, these techniques are applied after deep features have been extracted by the DCNN, rather than directly on the raw images. This approach aims to expand the representation of minority classes in the feature space. The methods explored include ROS, SMOTE, ADASYN, Borderline-SMOTE, SMOTE-NC, SMOTE-ENN, and SVM-SMOTE.
- Classification: After over-sampling, a Support Vector Machine (SVM) classifier is trained on the augmented feature set to perform the final defect classification.
Datasets
The paper introduces a large, newly built printing defect dataset:
- Coarse-grained dataset: Contains 15,307 defect samples across four categories: light defect (LD), dark defect (DD), knife wire defect (KD), and color deviation (CD). This dataset exhibits class imbalance, particularly with the CD class.
- Fine-grained dataset: Includes eleven types of printing defects, with 17,604 samples. This dataset is even more imbalanced than the coarse-grained one, making it a more challenging benchmark. It expands on the coarse-grained categories by adding crystal (CT), point (PT), no printing (NP), smears (SS), overprint (OP), and trailing (TL), and further subdividing the dark defects into paper powder (PP) and ink (IK).
All defect samples were collected from actual printing products in a factory setting.
Key Insights and Ideas
- Feature-Level Over-sampling is Key: Applying over-sampling techniques to extracted deep features rather than raw images is a critical insight. This allows for more targeted augmentation in a more discriminative feature space, proving more effective than image-level augmentation for this task.
- Custom DCNN Outperforms Transfer Learning: The authors' proposed DCNN architecture, designed specifically for printing defects, demonstrates superior feature extraction capabilities compared to several well-known pre-trained deep models (VGG, ResNet, Inception, Xception) when evaluated on the coarse-grained dataset.
- SMOTE-ENN Excels on Coarse-Grained Data: Among the tested over-sampling methods, SMOTE-ENN showed the best performance on the coarse-grained dataset, significantly improving accuracy.
- Over-sampling is Essential for Fine-Grained Data: The fine-grained dataset, with its extreme class imbalance, highlights the necessity of over-sampling. Without it, classification accuracy for minority classes plummets.
Key Results
- Coarse-Grained Dataset: The proposed deep model achieved a 96.86% classification accuracy without over-sampling, outperforming several transfer learning models.
- Fine-Grained Dataset: When combined with the SVM-SMOTE over-sampling method, the accuracy rate improved by over 20% compared to the method without over-sampling.
- Overall Performance: The paper demonstrates that the proposed deep model, when combined with appropriate over-sampling techniques (like SMOTE-ENN or SVM-SMOTE), can achieve high classification accuracies (e.g., average accuracy of 98.78% with SMOTE-ENN on the coarse-grained dataset and 94.37% with SVM-SMOTE on the fine-grained dataset).
- ROC Analysis: Receiver Operating Characteristic (ROC) curves and Area Under Curve (AUC) values confirmed the effectiveness of the over-sampling methods, with AUC values often reaching "very good" or "excellent" levels (e.g., AUC of 0.99 for SMOTE-ENN on the coarse-grained dataset).
Limitations and Future Work
- Execution Time: The authors acknowledge that the execution time of the proposed method needs to be considered to ensure it can keep pace with high-speed printing presses.
- Generalization: While the method shows promise, further work is needed to develop more advanced, robust, and generalized deep learning methods for improved classification performance.
- Exploration of Other Techniques: The paper focuses on over-sampling. Future work could explore other imbalance learning strategies or hybrid approaches.
- Dataset Size: Although a large dataset was built, the inherent difficulty in collecting defect samples means that further expansion and diversity could always be beneficial.
Summary updated