Cognitive Visual Inspection Service for LCD Manufacturing Industry

Yuanyuan Ding, Junchi Yan, Guoqiang Hu, Jun ZhuarXiv 2021 · OpenReview

TL;DR

This paper introduces a novel cognitive visual inspection system for LCD manufacturing that combines a robust defect recognition model with a flexible service architecture. By leveraging a hybrid approach of traditional computer vision and deep learning, the system achieves high accuracy in defect detection, classification, and impact evaluation, even with limited image-level annotations and varying image quality.

What Problem Does This Paper Address?

The flat-panel display (FPD) industry, particularly LCD manufacturing, faces a critical need for efficient and accurate quality control. Traditional manual inspection is labor-intensive and error-prone. While Automatic Optical Inspection (AOI) systems exist, they often rely on handcrafted features, making them sensitive to image acquisition variations and tightly coupled to specific hardware. This leads to high false alarm rates, difficulty in scaling, and limited capabilities in defect classification and impact assessment. The paper aims to address these limitations by developing a more robust, adaptable, and intelligent visual inspection system.

Methods

The proposed system is built on two core components: a robust defect recognition model and a cognitive visual inspection service architecture.

Defect Recognition Model: * Hybrid Approach: Combines conventional computer vision techniques with Deep Convolutional Neural Networks (DCNNs). * Defect Patch Identification: Uses a DCNN-based binary classifier trained with weakly supervised learning and a periodic pattern-based approach to identify potential defect regions within constant-size image patches. This method semi-automatically generates patch-level annotations, reducing the need for extensive manual labeling. * Self-Reference Based Defect Segmentation: A novel approach that uses a defect-free portion of the same inspection image as a reference template for segmentation. This makes the process robust to variations in image quality (brightness, sharpness) and avoids the need for offline template matching or pixel-level annotations. * Background-Aware Defect Classification: Employs a DCNN classifier that considers not only the defect patch but also its corresponding background. This is achieved by augmenting the input with background information (e.g., as an additional channel), helping to decouple defect features from background patterns and improve classification accuracy, especially for a large number of defect classes.

Cognitive Visual Inspection Service Architecture: * Decoupled Design: Separates defect recognition functions from front-end acquisition devices to a back-end service. * Modular Components: Consists of computing nodes (GPU-equipped servers), an inspection engine (managing model instances), and a controller (job dispatcher for routing requests and managing models). * Scalability and Flexibility: Designed for easy migration and up-scaling, allowing for diverse inspection models to be deployed for different product models and production stages.

Datasets

The paper utilizes a large-scale, real-world LCD dataset from a manufacturing line. This dataset includes images from different panel layers and products, with a significant number of defect categories (over 20) and a large sample size (hundreds of thousands). The authors split this data into training, testing, and validation sets with a ratio of 8:1:1.

Key Insights and Ideas

  • Weakly Supervised Learning for Annotation Efficiency: The paper demonstrates that combining weakly supervised learning with periodic pattern analysis can significantly reduce the burden of manual annotation for defect detection, making the training process more economical.
  • Self-Reference for Robust Segmentation: Using an image's own defect-free regions as a reference for segmentation is a clever way to overcome image quality variations without relying on external templates, which are often brittle.
  • Background Awareness in Classification: Recognizing that background patterns can interfere with defect classification, especially with many similar defect types, the paper's approach of incorporating background information as an input channel is a key insight for improving discriminative power.
  • Service Architecture for Adaptability: Decoupling the inspection logic into a service architecture allows for greater flexibility in deploying and managing inspection models, crucial for the dynamic nature of manufacturing environments.

Key Results

  • The proposed defect recognition model achieves high inspection accuracy.
  • The background-aware defect classification models consistently outperform the vanilla RGB network, with the RGB+G (adding grayscale background as a fourth channel) model offering a good balance of efficiency and accuracy.
  • The system has been successfully deployed in a real-world LCD manufacturing line, demonstrating its practical utility and scalability.
  • The paper reports accuracy metrics for various defect types and channel configurations, showing improvements with the proposed methods. For instance, overall accuracy for Layer X is reported to be higher than Layer Y.

Limitations and Future Work

  • Defect Categorization Bias: The paper notes that defect categorization is often based on manufacturing root causes rather than purely visual appearance, which can lead to similar-looking defects being difficult to distinguish purely visually. This suggests a limitation of appearance-based classification.
  • Cold-Start Problem and Domain Adaptation: As a learning-based model, it requires a reasonable amount of labeled data. The authors suggest domain adaptation as a future direction to repurpose learned models for new products or categories with limited data.
  • Incremental Learning of New Defects: The system currently doesn't inherently handle the emergence of new defect categories over time. Future work could explore incremental learning techniques or open-set CNN models to address this.

Summary updated

Abstract

With the rapid growth of display devices, quality inspection via machine vision technology has become increasingly important for flat-panel displays (FPD) industry. This paper discloses a novel visual inspection system for liquid crystal display (LCD), which is currently a dominant type in the FPD industry. The system is based on two cornerstones: robust/high-performance defect recognition model and cognitive visual inspection service architecture. A hybrid application of conventional computer vision technique and the latest deep convolutional neural network (DCNN) leads to an integrated defect detection, classfication and impact evaluation model that can be economically trained with only image-level class annotations to achieve a high inspection accuracy. In addition, the properly trained model is robust to the variation of the image qulity, significantly alleviating the dependency between the model prediction performance and the image aquisition environment. This in turn justifies the decoupling of the defect recognition functions from the front-end device to the back-end serivce, motivating the design and realization of the cognitive visual inspection service architecture. Empirical case study is performed on a large-scale real-world LCD dataset from a manufacturing line with different layers and products, which shows the promising utility of our system, which has been deployed in a real-world LCD manufacturing line from a major player in the world.
BibTeX
@misc{ding2021cognitive,
  author = {Yuanyuan Ding and Junchi Yan and Guoqiang Hu and Jun Zhu},
  title = {{Cognitive Visual Inspection Service for LCD Manufacturing Industry}},
  year = {2021},
  eprint = {2101.03747},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2101.03747},
}
Figure