Self-Supervised Vision Transformers for Scalable Anomaly Detection over Images

Stefano Samele, Matteo MatteucciIJCNN 2024 · OpenReview

Abstract

In this work, we leverage self-supervised pre-trained Vision Transformers (ViTs) and their self-attention maps to build a novel anomaly detection algorithm. The method, SAMSAD (Self-Attention MapS for Anomaly Detection), leverages the ability of pre-training procedures to obtain implicit images’ semantic segmentation masks. From this organization of features, a three-step procedure is proposed based on fine-tuning, clustering, and anomaly scoring. The algorithm achieves state-of-the-art performances of anomaly detection and segmentation in industrial product images, scaling to higher-resolution images with more effectiveness than its competitors.
BibTeX
@inproceedings{samele2024self,
  author = {Stefano Samele and Matteo Matteucci},
  title = {{Self-Supervised Vision Transformers for Scalable Anomaly Detection over Images}},
  booktitle = {IJCNN},
  year = {2024},
  url = {https://openreview.net/forum?id=4zVY7mpzIw},
}

Loading related content...

Figure