Abstract
Zero-shot Anomaly Detection (ZSAD) is a pivotal challenge in industrial inspection, aiming to detect defects in novel product categories without prior exposure. Current state-of-the-art approaches typically leverage vision-language models like CLIP, which necessitate extensive and often manual "prompt engineering." We introduce DZAD (Diffusion-based Zero-shot Anomaly Detection), the first framework to utilize generative diffusion models for anomaly detection without requiring any text prompts. By extracting semantic noise features across multiple denoising timesteps, DZAD effectively identifies anomalies by observing the model's internal generative process. Our method achieves state-of-the-art results on MVTec AD, VisA, and BTAD benchmarks, significantly outperforming existing prompt-based and prompt-free methods.
Introduction
The "holy grail" of industrial inspection is a model that can identify defects in products it has never seen before. While vision-language models have made strides in this area, their reliance on hundreds of manually designed text descriptions—to align visual features with concepts like "scratched" or "broken"—creates a significant bottleneck. DZAD addresses this by tapping into the rich semantic information generated during the "denoising" process of diffusion models. The core insight is that a generative model's difficulty in mapping a region back to a "normal" state serves as a powerful signal for anomaly detection.
Methodology
DZAD is built upon a Latent Diffusion Model (LDM) and introduces three primary innovations to repurpose generative features for discriminative tasks.
Semantic-Guided (SG) Diffusion
A common issue with standard diffusion models in anomaly detection is "semantic loss," where the model might reconstruct a defective object as a different, non-defective object (e.g., turning a broken nut into a perfect bolt). The SG network ensures the model maintains the core identity of the object while attempting to "fix" anomalous regions, preserving the necessary context for detection.
Multi-Timestep Feature Extraction
Instead of analyzing the final reconstructed image, DZAD examines the noise predicted by the model at various stages ($t$) of the denoising process. Anomalous regions produce distinct noise patterns. The authors discovered that for optimal detection, the importance of these features should increase slightly over time to capture subtle structural defects, leading to the proposal of a custom weighting function: $w_t = 1 + \exp(-3 \cdot (10 - n))$.
Feature Fusion
The framework compares features from the diffusion process with a pre-trained ResNet50 backbone across multiple scales. This dual-branch approach allows for the generation of high-resolution anomaly maps that precisely localize defects.
Experimental Results
DZAD was evaluated on three gold-standard benchmarks: MVTec AD, VisA, and BTAD. The model was tested in a true zero-shot fashion, ensuring no overlap between training and testing categories.
As shown in the table above, DZAD achieves state-of-the-art (SOTA) results across the board. On MVTec AD, it reached an image-level AUROC of 0.935, surpassing the previous best prompt-based method, AnomalyCLIP (0.911). On the VisA dataset, it achieved 0.902 Image AUROC, a significant improvement over AdaCLIP (0.858). Qualitatively, DZAD produces cleaner and more precise heatmaps than previous methods, especially on complex textures where other models often produce noisy detection maps.
Conclusion and Future Work
DZAD demonstrates that generative diffusion models can outperform vision-language models in zero-shot anomaly detection without the need for complex prompt engineering. While the model shows exceptional performance, challenges remain in specific categories like "Cable" due to high intra-class variability. Furthermore, the computational intensity of diffusion models presents a hurdle for real-time deployment. Future work will focus on optimizing these models for speed and enhancing their ability to handle highly diverse anomaly types.
Summary updated