Crane: Context-Guided Prompt Learning and Attention Refinement for Zero-Shot Anomaly Detections

Alireza Salehi, Mohammadreza Salehi, Reshad Hosseini, Cees G. M. Snoek, Makoto Yamada, Mohammad SabokrouarXiv 2025 · DOI · arXiv

Abstract

Zero-shot anomaly detection (ZSAD) aims to identify and localize anomalies in images without prior exposure to domain-specific training data. While foundation models like CLIP have shown promise, they often suffer from spatial misalignment and local blindness. We present Crane, a novel framework that enhances CLIP's spatial awareness through context-guided prompt learning and a refined attention mechanism. Crane achieves significant performance gains across 14 industrial and medical datasets, improving both image-level classification and pixel-level localization.

Introduction

Anomaly detection (AD) is critical in fields ranging from industrial quality control to medical diagnostics. Traditional supervised methods require large amounts of labeled anomalous data, which is often scarce. Zero-shot approaches leverage large-scale vision-language models like CLIP to generalize to unseen domains. However, CLIP's global training objective leads to "blurry" spatial features, making it difficult to pinpoint small, local defects. Crane addresses these limitations by refining how the model processes spatial information and aligns text prompts with image context.

Methods

Crane introduces three primary architectural enhancements to the standard CLIP pipeline to improve zero-shot performance without compromising general knowledge.

The Crane architecture features an Adapted Vision Encoder with Extended Self-Correlation Attention (E-Attn), Context-Guided Prompt Learning, and a Local-to-Global Fusion module.

Extended Self-Correlation Attention (E-Attn)

To fix the spatial shifting common in CLIP features, Crane replaces the final layers of the vision encoder with a self-correlation mechanism. This forces the model to focus on internal patch relationships, preserving better spatial alignment for precise heatmap generation.

Context-Guided Prompt Learning

Standard learnable prompts are static. Crane makes prompts "image-aware" by conditioning them on the global context ([CLS] token) of the input image. This allows the text encoder to generate embeddings that are more discriminative for the specific object being inspected.

Local-to-Global Fusion and Crane+

Crane utilizes a score-based pooling mechanism to ensure that small local anomalies are not lost in the global image representation. An enhanced version, Crane+, further integrates DINOv2 to provide superior spatial geometry, using it as a "spatial stabilizer" alongside CLIP's semantic logic.

Results and Discussion

The framework was evaluated on 14 diverse datasets, including industrial (e.g., MVTec-AD, VisA) and medical (e.g., ISIC, BrainMRI) domains.

Radar charts comparing Crane against AdaCLIP and AnomalyCLIP across multiple datasets, alongside qualitative localization results showing Crane's superior precision.

Crane consistently outperformed state-of-the-art baselines. In industrial settings, it achieved a 2.4% improvement in image-level AUROC and a 5.7% boost in pixel-level AUPRO. Notably, in medical domains where it had zero prior exposure, Crane+ improved localization by 5.0%. Despite these additions, the model maintains high efficiency, operating at approximately 3.7 FPS on an A6000 GPU.

Conclusion

Crane demonstrates that refining the spatial and contextual alignment of foundation models can significantly advance zero-shot anomaly detection. By bridging the gap between global semantic understanding and local spatial precision, Crane sets a new benchmark for generalizable anomaly detection. Future work will focus on narrowing the performance gap with full-shot models and improving the handling of complex logical anomalies.

Summary updated

Abstract

Zero-shot anomaly detection and localization aims to learn from source-domain data and generalize to unseen target domains without target-domain samples. Recent CLIP-based methods perform inference by comparing visual features with normal and abnormal textual prototypes; however, dense localization remains substantially weaker than image-level detection, especially for small or subtle abnormal regions. We identify two key limitations behind this gap: CLIP's vision encoder is primarily optimized for global image-text alignment, which limits its ability to preserve fine-grained spatial details, and the learned normal/abnormal text representations are not sufficiently aligned with dense visual features. To address these limitations, we propose Crane (Context-Guided Prompt Learning and Attention Refinement), a CLIP-based framework that first adapts the vision encoder with a correlation-based attention module to better preserve local visual structure. Second, it conditions learnable normal and abnormal prompts on global image context during training, improving instance-aware text-vision alignment. Third, it incorporates anomaly-aware local-to-global fusion, which injects anomaly-relevant patch features into the global image representation for more sensitive image-level detection. Finally, we show that the same correlation-based attention design can exploit spatial correlations from DINOv2, yielding a boosted variant, Crane+, with stronger localization ability. Across seven industrial benchmarks, Crane improves mean image-level AP by 4.5% over the strongest compared baseline, while Crane+ improves mean pixel-level AUPRO by 9.0%. The code is available at https://github.com/Alireza99Salehi/Crane.
BibTeX
@misc{salehi2025crane,
  author = {Alireza Salehi and Mohammadreza Salehi and Reshad Hosseini and Cees G. M. Snoek and Makoto Yamada and Mohammad Sabokrou},
  title = {{Crane: Context-Guided Prompt Learning and Attention Refinement for Zero-Shot Anomaly Detections}},
  year = {2025},
  eprint = {2504.11055},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2504.11055},
}
Figure