TL;DR
This paper introduces ADClick, a novel interactive image segmentation method that significantly reduces the cost of labeling industrial anomalies by generating high-quality pixel-level masks with just a few user clicks and language prompts. This approach not only achieves state-of-the-art performance in anomaly detection and localization but also paves the way for more efficient and accurate industrial inspection systems.
What Problem Does This Paper Address?
Industrial anomaly detection (AD) is crucial for quality control, but manually labeling anomalous pixels is extremely time-consuming and expensive. This has led many AD methods to rely on "one-class" classifiers trained only on anomaly-free data, which can limit their performance. While incorporating real defect samples improves accuracy, it exacerbates the labeling burden. This paper tackles this challenge by proposing a method that balances AD accuracy with reduced labeling costs, making it more practical for real-world industrial applications.
Methods
The core of the proposed solution is ADClick, an Interactive Image Segmentation (IIS) algorithm. ADClick leverages three key components to generate precise anomaly masks:
- Human Clicks: Users provide sparse annotations (3-5 clicks) indicating anomalous or normal regions.
- Language Prompts: Defect-specific textual descriptions guide the segmentation process, providing semantic context.
- Location-Aware Residual Features: The method extracts novel residual features that are more stable and generalize better than raw pixels. These features are derived by matching patches in the query image to similar patches in anomaly-free reference images, considering spatial proximity.
ADClick uses these inputs within a Swin transformer backbone to predict dense pixel labels. The paper also introduces ADClick-Seg, an extension designed for anomaly detection and localization. ADClick-Seg is trained using the "pseudo labels" generated by ADClick, effectively leveraging the weak labels for improved performance.
Datasets
The proposed methods were evaluated on two well-established industrial anomaly detection datasets:
- MVTec AD: A widely used benchmark for anomaly detection and localization.
- KolektorSDD2: Another dataset for anomaly detection tasks.
Key Insights and Ideas
- Bridging the Gap: The paper successfully bridges the gap between the need for accurate anomaly detection and the prohibitive cost of manual pixel-level labeling.
- Power of Weak Supervision: By using interactive segmentation with minimal user input, ADClick demonstrates that high-quality labels can be generated efficiently, making supervised AD more accessible.
- Synergy of Modalities: The combination of sparse human clicks, semantic language prompts, and robust residual features proves to be highly effective, outperforming existing IIS methods.
- Location-Awareness is Key: Incorporating image-similarity constraints and location-aware matching for residual feature extraction leads to improved generalization and accuracy.
- IIS for AD Labeling: This work is the first to apply the concept of Interactive Image Segmentation specifically for generating training labels in anomaly detection tasks.
Key Results
- Label Generation: ADClick generates high-quality anomaly labels with only 3-5 manual clicks per image, achieving an Average Precision (AP) of 94.1% on the MVTec AD dataset. It significantly outperforms existing state-of-the-art IIS methods.
- Anomaly Detection and Localization: The ADClick-Seg model, trained with ADClick's pseudo labels, achieves state-of-the-art performance in supervised AD tasks.
- On MVTec AD, it reaches an AP of 86.4%.
- On KolektorSDD2, it achieves an AP of 78.4% and a PRO of 98.6%.
- Unsupervised Performance: Even in an unsupervised setting (without linguistic features), ADClick-Seg demonstrates competitive performance, tying with SOTA methods in PRO and Image-AUROC on MVTec AD.
Limitations and Future Work
- Manual Prompt Refinement: While ChatGPT is used for prompt generation, the paper notes that annotators occasionally need to delete invalid descriptions. Automating this prompt validation through an NLP classifier is suggested as future work.
- mIoU in 5-click test: The paper acknowledges a slight dip in mIoU numbers for the 5-click test compared to some methods, attributing it to the lack of a "zoom in" technique. They suggest that a more sophisticated inference scheme could improve this.
- Interactive Annotation Tool: The authors have developed an interactive annotation tool based on ADClick, which they plan to make publicly available, encouraging further adoption and research.
- Further Exploration: The paper highlights the need for further exploration into the unique nature of AD supervision to achieve even more efficient AD labels and better AD results.
Summary updated