Reliable fabric defect detection via Bayesian uncertainty modeling

Wai Keung Wong, Zhewei Chen, Jiahua Liao, Ying QuJournal of Textile Engineering & Fashion Technology 2024 · DOI

TL;DR

This paper introduces a Bayesian U-Net model for fabric defect detection that quantifies prediction uncertainty. By using Variational Inference and Monte Carlo sampling, the model provides reliable confidence estimates, improving F1 scores by 2-4% and enabling better decision-making for quality control inspectors.

What Problem Does This Paper Address?

Deep learning models have shown great promise in detecting defects in textiles, but they often suffer from overconfidence in their predictions. This is particularly problematic when encountering new defect types not seen during training or when dealing with imperfectly labeled data. This overconfidence makes it difficult for human inspectors to know when to trust the model's output and when manual verification is necessary, hindering real-world adoption. The paper aims to solve this by providing a model that not only detects defects but also reliably indicates its own uncertainty.

Methods

The core of this work is the integration of Bayesian principles into the popular U-Net architecture for semantic segmentation. Specifically, the authors employ Variational Inference (VI) to approximate the posterior distribution of the model's weights. During training, VI transforms the complex task of inferring weight distributions into a more manageable optimization problem.

For inference, the model uses Monte Carlo (MC) sampling. This involves performing multiple forward passes through the network with different sampled weights. This process generates several predictions for each pixel, allowing for the calculation of three key uncertainty metrics:

  • MC sample variance: Measures the variability of predictions across different samples.
  • Predictive entropy: Quantifies the informational content or uncertainty in the model's prediction for a given pixel.
  • Mutual information: Assesses the shared information between the model's posterior and predictive distributions.

These metrics are visualized as per-pixel uncertainty maps, providing a detailed understanding of where the model is less confident.

Datasets

The proposed method was evaluated on two publicly available datasets:

  • Fabric Stain dataset: Initially annotated with bounding boxes, this dataset was then refined with pixel-level annotations created by experts.
  • AITEX dataset: This dataset already provides pixel-level annotations and features a diverse range of defect types.

Both datasets were split into training (60%), validation (20%), and testing (20%) sets.

Key Insights and Ideas

  • Uncertainty as a proxy for reliability: The study reveals a strong negative correlation between prediction uncertainty (specifically logits variance) and segmentation performance (F1 score). This means higher uncertainty generally corresponds to lower accuracy, providing a valuable signal for quality control.
  • Ensemble effect of Bayesian inference: By sampling weights, the Bayesian U-Net effectively acts like an ensemble of models. This ensemble approach leads to more robust predictions and better generalization, especially in complex or ambiguous regions.
  • Decision support for inspectors: The uncertainty maps provide actionable insights. High uncertainty regions can be automatically flagged for human review, streamlining the inspection process and ensuring that human expertise is applied where it's most needed.
  • Robustness to label noise: The probabilistic nature of the Bayesian model, particularly the averaging effect of MC sampling, makes it inherently more resilient to noisy or incorrect annotations in the training data.

Key Results

  • Improved Segmentation Performance: The Bayesian U-Net achieved a significant improvement in F1 score, with a 2-4% increase compared to the baseline frequency-domain U-Net model across both datasets. Accuracy and Intersection over Union (IOU) metrics also saw notable gains.
  • Quantifiable Uncertainty: The model successfully generated per-pixel uncertainty maps, visualizing the model's confidence levels.
  • Correlation Analysis: A strong negative correlation (r = -0.649 for Stain, r = -0.777 for AITEX) was observed between logits variance and F1 scores, confirming the utility of uncertainty as a performance indicator.
  • Qualitative Improvements: Visual examples demonstrate that the Bayesian U-Net can better delineate defect boundaries and provide more nuanced confidence estimates, especially in challenging areas.

Limitations and Future Work

  • Computational Cost: A primary limitation acknowledged is the substantial resource consumption and extended inference time associated with Monte Carlo sampling, which can be a barrier for real-time applications or resource-constrained platforms.
  • Future Directions: The authors propose researching more efficient Bayesian models and inference methods to reduce computational overhead while retaining the benefits of uncertainty estimation. Optimizing these models for faster performance is key to expanding their applicability to real-time scenarios.

Summary updated

Abstract

Despite the demonstrated capability of deep learning models in detecting anomalies in textile images, their predictions in real-world applications tend to be overly confident, especially when faced with defect types not previously encountered in the training set or when dealing with low-quality annotations. This excessive confidence in predictions limits the practical application of deep learning methods in textile defect detection, as it fails to provide inspectors with reliable guidance on when to trust the model's predictions and when manual verification is necessary. To address this issue, this paper introduces a Bayesian fabric anomaly detection model that utilizes Variational Inference (VI) to apply Bayesian inference to the widely used U-Net architecture. During the inference phase, the model employs Monte Carlo sampling to perform multiple forward passes, generating three types of uncertainty estimations and per-pixel uncertainty maps, thus providing comprehensive evidence for decision-making. This method not only estimates the uncertainty of model predictions but also improves the F1 score by 2-4% over the baseline U-Net model in the frequency domain. This study proves the Bayesian approach boosts fabric anomaly detection and decision-making by optimizing model performance and reducing reliance on inaccurate predictions.
BibTeX
@article{wong2024reliable,
  author = {Wai Keung Wong and Zhewei Chen and Jiahua Liao and Ying Qu},
  title = {{Reliable fabric defect detection via Bayesian uncertainty modeling}},
  journal = {Journal of Textile Engineering \& Fashion Technology},
  year = {2024},
  doi = {10.15406/jteft.2024.10.00371},
}
Figure