Prior Normality Prompt Transformer for Multi-class Industrial Image Anomaly Detection

Haiming Yao, Yunkang Cao, Wei Luo, Weihang Zhang, Wenyong Yu, Weiming Shen

Abstract

Image anomaly detection plays a pivotal role in industrial inspection. Traditional approaches often demand distinct models for specific categories, resulting in substantial deployment costs. This raises concerns about multi-class anomaly detection, where a unified model is developed for multiple classes. However, applying conventional methods, particularly reconstruction-based models, directly to multi-class scenarios encounters challenges such as identical shortcut learning, hindering effective discrimination between normal and abnormal instances. To tackle this issue, our study introduces the Prior Normality Prompt Transformer (PNPT) method for multi-class image anomaly detection. PNPT strategically incorporates normal semantics prompting to mitigate the "identical mapping" problem. This entails integrating a prior normality prompt into the reconstruction process, yielding a dual-stream model. This innovative architecture combines normal prior semantics with abnormal samples, enabling dual-stream reconstruction grounded in both prior knowledge and intrinsic sample characteristics. PNPT comprises four essential modules: Class-Specific Normality Prompting Pool (CS-NPP), Hierarchical Patch Embedding (HPE), Semantic Alignment Coupling Encoding (SACE), and Contextual Semantic Conditional Decoding (CSCD). Experimental validation on diverse benchmark datasets and real-world industrial applications highlights PNPT's superior performance in multi-class industrial anomaly detection.

The Challenge of Multi-class Anomaly Detection

In industrial settings, anomaly detection (AD) has traditionally relied on a "one-class-one-model" paradigm. While effective, maintaining separate models for every product—from screws to transistors—creates a significant deployment and maintenance burden. The research community is shifting toward unified (multi-class) models, where a single model handles multiple object categories simultaneously.

However, unified models face unique challenges. As shown in the learning distribution comparison below, separate models ($M_1, M_2, M_3$) maintain distinct decision boundaries. A unified model ($M_U$) must learn a much more complex distribution, often leading to overlap between normal and abnormal features across different classes.

Comparison of training paradigms and learning distributions

Overcoming the Identical Mapping Trap

Beyond distribution overlap, reconstruction-based models often fall into the "identical mapping" trap. In this scenario, the model becomes so proficient at reconstructing inputs that it reconstructs anomalies (e.g., scratches or missing components) just as perfectly as normal features. This results in a near-zero reconstruction error, rendering the anomaly invisible to the system.

Existing solutions like "Explicit normality Storage" (memory banks) attempt to solve this by matching features against stored normal samples, but they often suffer from "Unaligned Match" issues where the retrieved feature doesn't perfectly correspond to the spatial location of the test patch.

Comparison of method motivations: Implicit vs. Explicit normality

The PNPT Framework

The Prior Normality Prompt Transformer (PNPT) addresses these issues by treating normality as an explicit "prompt" that guides the reconstruction process. Instead of a single stream, PNPT uses a dual-stream architecture that combines the test image's features with a reference prompt of what a normal version of that object should look like.

Key Architectural Modules

  • Class-Specific Normality Prompting Pool (CS-NPP): Rather than storing millions of raw patches, CS-NPP stores a compact, learned "normality feature" for each category. This is more memory-efficient and provides a stable semantic baseline.
  • Hierarchical Patch Embedding (HPE): A multi-scale strategy ensures the model captures both fine-grained structural defects (like micro-cracks) and large-scale logical defects (like missing components in an assembly).
  • Semantic Alignment Coupling Encoding (SACE): This module uses cross-attention to align the test image features with the normality prompt, effectively highlighting discrepancies between the input and the "ideal" normal state.
  • Contextual Semantic Conditional Decoding (CSCD): The decoder is conditioned on the normal prompt, forcing it to reconstruct a normal version of the object even when the input is defective. This ensures a high reconstruction error for anomalies, making them easy to detect.

Experimental Results

PNPT was evaluated against rigorous benchmarks, including MVTec AD and MVTec LOCO AD, setting new state-of-the-art (SOTA) records for multi-class detection:

  • MVTec AD: Achieved 98.3% Image-level AUROC, outperforming the previous UniAD baseline by +1.8%.
  • MVTec LOCO: Achieved an overall AUROC of 81.9%, demonstrating superior performance in detecting complex logical anomalies where traditional models struggle.
  • Efficiency: The model maintains a high inference speed of 53.7ms per image and requires significantly less GPU memory (~5.7GB) compared to traditional memory-bank methods like PatchCore (~12-37GB) in multi-class settings.

Conclusion

PNPT demonstrates that explicit normality prompting is a powerful tool for overcoming the identical mapping shortcut in unified anomaly detection. By guiding reconstruction with prior knowledge, the model achieves high accuracy and efficiency. Future work will explore the integration of Large Vision-Language Models (LVLMs) to incorporate natural language reasoning into the detection of complex logical constraints.

Summary updated

Abstract

Image anomaly detection plays a pivotal role in industrial inspection. Traditional approaches often demand distinct models for specific categories, resulting in substantial deployment costs. This raises concerns about multi-class anomaly detection, where a unified model is developed for multiple classes. However, applying conventional methods, particularly reconstruction-based models, directly to multi-class scenarios encounters challenges such as identical shortcut learning, hindering effective discrimination between normal and abnormal instances. To tackle this issue, our study introduces the Prior Normality Prompt Transformer (PNPT) method for multi-class image anomaly detection. PNPT strategically incorporates normal semantics prompting to mitigate the "identical mapping" problem. This entails integrating a prior normality prompt into the reconstruction process, yielding a dual-stream model. This innovative architecture combines normal prior semantics with abnormal samples, enabling dual-stream reconstruction grounded in both prior knowledge and intrinsic sample characteristics. PNPT comprises four essential modules: Class-Specific Normality Prompting Pool (CS-NPP), Hierarchical Patch Embedding (HPE), Semantic Alignment Coupling Encoding (SACE), and Contextual Semantic Conditional Decoding (CSCD). Experimental validation on diverse benchmark datasets and real-world industrial applications highlights PNPT's superior performance in multi-class industrial anomaly detection.
BibTeX
@article{yao2024prior,
  author = {Haiming Yao and Yunkang Cao and Wei Luo and Weihang Zhang and Wenyong Yu and Weiming Shen},
  title = {{Prior Normality Prompt Transformer for Multi-class Industrial Image Anomaly Detection}},
  journal = {IEEE Trans. Industrial Informatics},
  year = {2024},
  url = {https://openreview.net/forum?id=aHcURzB7zK},
}
Figure