Abstract
Identifying Customer Induced Damage (CID) is a key part in warranty programs of electronics manufacturers. CID is defined as any damage in the unit performed by an unauthorized person including the customer in a Printed Circuit Board (PCB). In such cases, damaged units are not covered by warranty. The inspection of CIDs is usually performed by humans which may be costly and error prone. Modern computer vision techniques for object detection using deep neural networks can automatically and accurately detect CIDs on PCBs. The training of such networks requires a large labeled dataset of image examples of CIDs. Daily, hardware factories and repair centers generate hundreds of unlabeled images. Labeling them manually is laborious and time-consuming. Therefore, it is crucial to label the minimum amount of images such that the trained neural network can achieve comparable accuracy as if it were trained with the whole dataset. To this end, we propose an active learning approach that selects the most informative images for the object detector. For that, our approach is based on the uncertainty of the object detector, i.e., it selects new images based on class probability distribution given by the object detector. Also, we tackle some challenges that are intrinsic to this problem: i) it is a multiclass object detection problem since there are many types of defects; ii) there is a class accuracy imbalance; iii) there is a focus on recall, e.g. false positives are less harmful than false negatives, and iv) there are many images with no object which should not be selected for labeling. We evaluate this approach by using it to iteratively sample data, train and evaluate a model, and compare it with randomly sampled data. The results show that our method consistently outperforms random sampling by an average margin of 21.6%, proving to be a viable alternative for reducing the labeling cost and increasing detection accuracy in this domain.
Introduction
When a computer is returned for repair, manufacturers must distinguish between factory defects and Customer Induced Damage (CID). CID encompasses physical damage such as scratches on the PCB or bent pins in a CPU socket. Currently, manual inspection is the standard, but it is slow, expensive, and prone to human error. While deep learning models can automate this process, they require vast amounts of labeled data. In high-volume repair centers, thousands of images are generated daily, but manually labeling every image is inefficient, especially since many images contain no damage. The challenge lies in identifying the smallest subset of images that, when labeled, provides the maximum improvement in model accuracy.
The CUSAL Methodology
To address this, researchers developed Categorical Uncertainty Sampling Active Learning (CUSAL). This framework operates in an iterative loop designed to maximize the information gain from each human-labeled image.
The CUSAL process consists of two primary stages:
- Image Scoring: Rather than assigning a single uncertainty score to an entire image, CUSAL calculates "categorical uncertainty." It aggregates the uncertainty of all predicted objects within an image, grouped by class. The image’s final priority score is determined by the highest uncertainty found among its classes, ensuring the model prioritizes images where it is most "confused" about specific defect types.
- Image Selection: To mitigate class imbalance, the system calculates a "difficulty score" for each class based on its current recall (1 - Recall). Classes that the model struggles to identify are allocated a higher labeling "budget" in subsequent rounds.
The underlying architecture utilizes a Mask R-CNN with a Swin Transformer backbone, providing a robust foundation for detecting and segmenting complex defects.
Dataset and Defect Classification
The study utilized a real-world dataset of 2,844 RGB images provided by a hardware manufacturer. The defects were categorized into three main classes:
- Class S: Scratches, abrasions, and nicks.
- Class P: Pin damage, such as bent or missing pins.
- Class D: Cracks, breaks, and distortions.
The dataset presented significant challenges, including noise from blurred images and a high percentage (~20%) of "clean" boards with no visible damage.
Key Results and Insights
Experimental results demonstrate that CUSAL significantly outperforms traditional random sampling across 16 iterations:
- Accuracy: CUSAL achieved an average margin of 21.6% in mAP (mean Average Precision) over the baseline.
- Recall: Overall recall improved by 39%, with the most difficult class (Pin damage) seeing a 60% improvement.
- Efficiency: By the peak of the study, 85% of the images selected by CUSAL for labeling contained actual defects, compared to the 77% baseline in the raw data pool. This effectively filters out "boring" data, allowing human experts to focus on informative samples.
Conclusion
CUSAL provides a model-agnostic strategy for reducing labeling costs while increasing detection accuracy in industrial defect inspection. By leveraging categorical uncertainty and recall-driven balancing, the system automatically focuses on the most difficult and informative samples. Future work will explore diversity maximization to prevent the model from becoming "blind" to rare, novel defect types that it currently assigns low probability scores.
Summary updated