Introduction
Traditional Anomaly Detection (AD) models are typically specialists, requiring extensive "normal" data for a specific object category to define what constitutes a defect. Consequently, a model trained to identify defects in one category, such as a "hazelnut," often fails when applied to a different category, like a "metal nut" or a "capsule." This limitation necessitates retraining for every new application, which is costly and time-consuming.
PromptAD addresses this by introducing a Zero-Shot Anomaly Detection (ZSAD) framework. By leveraging the multi-modal capabilities of CLIP, PromptAD uses natural language prompts to describe both "normal" and "abnormal" states, allowing it to generalize to entirely new object categories without additional training or visual examples.
The PromptAD Framework
PromptAD is built upon a frozen CLIP backbone and utilizes a dual-branch architecture to bridge the gap between visual information and semantic descriptions.
1. The Normality Branch
This branch establishes a "profile of normality" by using text prompts to define what a healthy object should look like. For instance, a prompt might describe "a short metal pin with a helical thread."
2. The Abnormality Branch
In contrast, the abnormality branch explicitly models anomalous behaviors using prompts like "crack," "scratch," or "hole." This allows the model to learn specific defect patterns that can be applied across different object types.
Enhancing Generalization through Interaction
To ensure these branches work synergistically and avoid overfitting to training classes, PromptAD introduces two key interaction components:
- Cross-view Contrastive Learning (CCL): This regularizes training by using information from one branch to refine the other, calculating residuals to ensure the model learns transferable features.
- Cross-view Mutual Interaction (CMI): This acts as a feedback loop. If the normality branch detects a deviation missed by the abnormality branch, it provides an attention map to guide the abnormality branch toward the neglected region.
The model is optimized using a Symmetric Deviation Loss, which enforces a statistically significant gap between the scores assigned to normal and anomalous pixels.
Experimental Results
The researchers evaluated PromptAD on major industrial benchmarks, including MVTec AD and AITEX, using a leave-one-category-out setup to test zero-shot generalization.
PromptAD significantly outperformed existing state-of-the-art baselines:
- Classification: On MVTec AD (1-shot), PromptAD achieved an average AUC of 0.908, compared to 0.815 for the previous best-performing method (DRA).
- Localization: For pixel-level anomaly localization, PromptAD reached a pAUROC of 0.921, surpassing WinCLIP (0.851).
- Efficiency: The model demonstrated high "data efficiency," where well-designed text prompts effectively replaced the need for hundreds of labeled images.
Conclusion
PromptAD demonstrates that high-quality text descriptions can compensate for missing visual data in anomaly detection. By modeling both normality and abnormality and facilitating their interaction, the framework achieves superior zero-shot performance. Future work will focus on automating prompt generation and improving robustness to complex, non-industrial backgrounds.
Summary updated