Print Defect Mapping with Semantic Segmentation

Augusto Valente, Cristina Wada, Deangela Neves, Deangeli Neves, Fabio Perez, Guilherme Megeto, Marcos Cascone, Otavio Gomes, Qian LinWACV 2020

TL;DR

This paper introduces the first end-to-end deep learning framework for pixel-level print defect mapping using semantic segmentation. By generating synthetic print defects and employing a modified DeepLab-v3+ architecture, the authors achieve robust defect detection and mapping, outperforming traditional methods.

What Problem Does This Paper Address?

Ensuring high print quality is crucial for customer satisfaction in the printing industry. Traditionally, identifying and mapping print defects (like streaks, banding, spots) relies on manual inspection, which is time-consuming, error-prone, and subjective. While automated methods exist, they often focus on coarse defect localization, target only one defect type, and require complex, hand-crafted features. This paper addresses the need for a comprehensive, automated solution that can map various print defects at a fine, pixel-level resolution.

Methods

The core of this work is a novel framework based on semantic segmentation, a computer vision technique that assigns a label to every pixel in an image. The authors leverage the DeepLab-v3+ architecture, known for its ability to capture both rich contextual information and sharp object boundaries, as the backbone of their model.

A significant contribution is their method for generating synthetic training data. Recognizing the difficulty and cost of manually annotating large datasets, they developed a pipeline that simulates:

  • Print defects: Specifically, dark streaks and color banding, by blending textures and manipulating color channels.
  • Print-scan effects: Mimicking the degradation and color shifts that occur when a printed document is scanned, using polynomial models derived from real device color transformations.

This synthetic data generation allows for cost-free, automated pixel-level annotation.

The framework supports two input modes:

  • No-Reference (NR): Uses only the defective printed image.
  • Full-Reference (FR): Uses both the original (defect-free) and the defective printed image.

For inference, they propose two strategies:

  • Resized Image: Downscales images for faster processing, yielding coarser results.
  • Patch-based: Processes the image in smaller patches for more detailed, fine-grained detection, but at a slower speed.

Datasets

The authors created and used two datasets:

  • Semi-synthetic dataset: Composed of 103 background images to which synthetic artifacts (banding and streaks) were added. These images were then printed and scanned at 600 dpi. This dataset allowed for multi-label banding (cyan, magenta, yellow, black) and provided automatic ground truth masks.
  • Real dataset: Consists of 30 scanned images with actual banding and streak defects collected from defective printers. These were also scanned at 600 dpi and manually annotated. For this dataset, banding defects were assigned a single label due to the difficulty of manual multi-channel annotation.

Key Insights and Ideas

  • Semantic segmentation is well-suited for print defect mapping: The pixel-level precision offered by semantic segmentation is a significant advantage over traditional coarse detection methods.
  • Synthetic data generation is a viable and cost-effective solution: The ability to create realistic print defects and print-scan effects bypasses the laborious and expensive process of manual data annotation.
  • No-Reference (NR) models can outperform Full-Reference (FR) models: Surprisingly, the NR approach achieved better results. The authors hypothesize that concatenating FR inputs might confuse the model, whereas the NR model, with a simpler input space, converges more effectively.
  • Patch-based inference yields superior quality: While slower, processing images in patches leads to significantly finer and more accurate defect detection compared to resizing the entire image.
  • Deep learning models can generalize from synthetic to real data: The framework demonstrates that knowledge learned from simulated defects can be effectively applied to identify real-world print imperfections.

Key Results

  • Quantitative Performance:
    • On the semi-synthetic dataset, the patch-based NR method achieved the highest mIoU of 0.45 ± 0.04.
  • On the real dataset, the patch-based NR method also performed best with an mIoU of 0.49 ± 0.04.
  • The NR method consistently outperformed the FR method across both datasets and inference strategies.
  • The resized image approach was significantly faster (around 1.7-2.5 seconds per image) compared to the patch-based approach (around 17.8-20.5 seconds per image).
  • Qualitative Findings:
    • The patch-based inference strategy produced noticeably finer-grained defect detection than the resized image approach.
  • The model was capable of detecting more subtle defects, including fading streaks, when compared to a baseline method from Wang et al. [18] (achieving an IoU of 0.299 vs. 0.183).

Limitations and Future Work

  • Limitations:
    • The authors acknowledge that human annotators can introduce biases, and achieving perfect agreement on defect segmentation can be challenging, especially for thin defects.
  • While the synthetic data is realistic, it may not perfectly capture all nuances of real-world defects.
  • Future Work:
    • Explore more advanced DCNN architectures and attention mechanisms to further improve performance.
  • Enhance the defect simulation pipeline and data augmentation techniques.
  • Incorporate a wider variety of defect types into the model.
  • Extend the framework to predict a global quality score alongside defect mapping, potentially using multi-task learning.
  • Investigate more relaxed metrics or methods to account for inter-annotator agreement issues.

Summary updated

BibTeX
@inproceedings{valente2020print,
  author = {Augusto Valente and Cristina Wada and Deangela Neves and Deangeli Neves and Fabio Perez and Guilherme Megeto and Marcos Cascone and Otavio Gomes and Qian Lin},
  title = {{Print Defect Mapping with Semantic Segmentation}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2020},
}
Figure