Real-Time Detection of Customer-Induced Damage in Printed Circuit Boards Using Mobile Devices and YOLO Detectors

J. Santiago, Victor A. E. Farias, Lucas Sena, João P. P. GomesLearning 2024 · DOI

Introduction

The electronics industry faces a significant challenge in managing warranty claims. Distinguishing between factory defects and Customer-Induced Damage (CID)—damage caused by unauthorized handling or misuse—is crucial for maintaining manufacturer revenue. Traditionally, this inspection is a manual process requiring skilled technicians to identify subtle scratches, bent pins, or structural distortions. This manual approach is time-consuming, expensive, and prone to human error.

To address these inefficiencies, researchers have developed a real-time, mobile-based detection system. By leveraging computer vision and deep learning, specifically the YOLO (You Only Look Once) object detection framework, the system enables technicians to perform on-site diagnostics directly on mobile devices without relying on cloud connectivity.

Methodology

The study evaluated sixteen experimental configurations across four lightweight YOLO architectures: YOLOv3-MobileNetV2, YOLOv6 (Lite and Standard), YOLOv7, and YOLOv8. The goal was to find a model that balances detection accuracy with the strict computational limits of mobile hardware.

Conceptual overview of a deep learning architecture for image processing.

The deployment pipeline involved several optimization steps:

  1. Training: Models were trained using PyTorch on high-performance NVIDIA V100 GPUs.
  2. Serialization: To ensure compatibility with mobile environments, models were converted to TorchScript, removing Python-specific dependencies.
  3. Mobile Optimization: The team utilized the ncnn framework, a high-performance inference engine optimized for mobile CPUs and GPUs.
  4. Application Development: A native Android app was developed using Kotlin and OpenCV-mobile to manage the real-time camera feed and model inference.

Dataset and Training

The models were trained on a proprietary dataset from a private electronics manufacturer's repair centers. The raw data consisted of approximately 82,000 unlabeled images, from which a labeled subset of 2,179 images was created. This subset contained 2,529 annotations across three primary damage classes:

  • Scratch (S): Surface-level abrasions on the PCB.
  • Pin damage (P): Deformed or broken pins within sockets.
  • Distortion/break (D): Physical warping or structural fractures of the board.

The researchers found that a high input resolution (1280x1280) was necessary to detect these often minute defects, despite the increased computational cost.

Key Results

The YOLOv6-N6 model emerged as the most effective solution for mobile deployment. While heavier models like YOLOv6-S6 offered slightly better accuracy, the N6 variant achieved a comparable [email protected] of 33.1% with significantly fewer resources—utilizing only 10.34M parameters and 49.57G FLOPs.

In real-world testing on a Samsung Galaxy A54 (Exynos 1380), the system achieved an average of 5.7 FPS using the mobile CPU. Interestingly, the CPU outperformed the mobile GPU (Mali-G68) in these tests, as the GPU encountered memory access bottlenecks that hindered performance.

Conclusion and Future Work

This research demonstrates the feasibility of deploying sophisticated object detection models on mid-range mobile devices for industrial quality control. While the current system provides a solid foundation, future work will focus on improving the detection of very small objects (currently <9% AP) and expanding the dataset using Active Learning techniques. Additionally, further optimization of mobile GPU kernels could potentially unlock higher frame rates, making the tool even more responsive for technicians in the field.

Summary updated

Abstract

The identification of Consumer-induced damage is essential for electronics manufacturers’ warranty programs. Consumer Induced Damage (CID) is any damage caused by an unauthorized person, including the consumer. The product warranty does not cover these damages, avoiding expenses in the manufacturer’s revenue. The consumer-induced damage warranty process is usually done manually by technically trained people. However, this task demands a lot of attention to detail, can be time-consuming, and is susceptible to human errors. With this in mind, this work presents an object detection model for low-computational-cost devices that uses computer vision and deep learning methods with YOLO detectors embedded in mobile devices to identify consumer-induced damages on printed circuit boards (PCB). We conducted sixteen experiments with four YOLO neural network architectures and successfully developed a mobile application for CID detection. Our best model achieved a [email protected] of 33.1% and an average of 5.7 FPS on real mobile devices.
BibTeX
@inproceedings{santiago2024real,
  author = {J. Santiago and Victor A. E. Farias and Lucas Sena and João P. P. Gomes},
  title = {{Real-Time Detection of Customer-Induced Damage in Printed Circuit Boards Using Mobile Devices and YOLO Detectors}},
  booktitle = {Learning},
  year = {2024},
  doi = {10.21528/lnlm-vol22-no1-art2},
}
Figure