Towards Adversarial Robustness and Uncertainty Quantification in DINOv2-based Few-Shot Anomaly Detection

Akib Mohammed Khan, Bartosz KrawczykarXiv 2025 · DOI · arXiv · OpenReview

Introduction

Few-Shot Anomaly Detection (FSAD) has seen significant advancements through the use of Vision Foundation Models (VFMs) like DINOv2. By extracting rich features and employing simple nearest-neighbor searches, these models can identify defects with high accuracy using minimal "normal" reference data. However, as these systems move into safety-critical industrial applications, two critical "trust gaps" emerge: adversarial vulnerability and a lack of uncertainty calibration. This paper investigates the fragility of DINOv2-based FSAD to imperceptible perturbations and proposes Platt scaling as a mechanism for both calibration and attack detection.

Methodology

The study utilizes AnomalyDINO, a training-free detector, as its baseline. To address the non-differentiable nature of k-Nearest Neighbor (k-NN) logic, the authors developed a novel "Probe" attack.

The "Probe" Attack

Since gradients cannot be easily backpropagated through a k-NN search, a lightweight linear "probe" head is attached to the frozen DINOv2 features. This probe is used to generate adversarial perturbations via the Fast Gradient Sign Method (FGSM). The resulting attacked image is then evaluated against the original k-NN detector.

The proposed 'Probe' attack pipeline using a linear classifier to generate adversarial perturbations for a frozen DINOv2 encoder.

Calibration and Uncertainty

To transform raw distance-based anomaly scores into interpretable probabilities, the researchers applied Platt scaling. This involves fitting a logistic regression model to a small validation set of scores. Furthermore, the calibrated probabilities allow for the calculation of predictive entropy, which serves as a measure of the model's uncertainty.

Workflow for Platt scaling calibration and the resulting predictive entropy distributions for clean versus adversarial inputs.

Experimental Results

The approach was evaluated on the MVTec-AD and VisA datasets, representing gold-standard benchmarks for industrial inspection.

Adversarial Vulnerability

Results indicate that DINOv2-based FSAD is highly susceptible to adversarial attacks. On MVTec-AD (4-shot), the AUROC plummeted from 97.55% to 59.68% under a standard FGSM attack. On the more complex VisA dataset, performance dropped to near-random levels (~52%). This suggests that tiny perturbations can effectively "push" anomaly embeddings into the "normal" cluster in feature space, causing confident misclassifications.

Calibration and Detection

Platt scaling significantly improved the reliability of anomaly scores without altering the ranking of results. In the 1-shot MVTec-AD setting, the Expected Calibration Error (ECE) was reduced from 0.4261 to 0.0536, making the scores far more representative of actual defect probabilities.

Crucially, predictive entropy was found to be a robust indicator of adversarial tampering. In the 1-shot setting, clean images exhibited a mean entropy of 0.12, whereas adversarial images showed a sharp increase to 0.51. This clear separation allows for the implementation of an entropy-based alarm system to flag potential attacks for human operators.

Conclusion

While DINOv2 provides a powerful foundation for FSAD, its feature space remains brittle. This work demonstrates that simple post-processing techniques like Platt scaling can mitigate interpretability issues and provide a vital signal for detecting adversarial attacks through predictive entropy. Future research will focus on more complex iterative attacks and developing geometry-aware defense mechanisms to enhance the inherent robustness of the feature space.

Summary updated

Abstract

Foundation models such as DINOv2 have shown strong performance in few-shot anomaly detection, yet two key questions remain unexamined: (i) how susceptible are these detectors to adversarial perturbations; and (ii) how well do their anomaly scores reflect calibrated uncertainty? Building on AnomalyDINO, a training-free deep nearest-neighbor detector over DINOv2 features, we present one of the first systematic studies of adversarial attacks and uncertainty estimation in this setting. To enable white-box gradient attacks while preserving test-time behavior, we attach a lightweight linear head to frozen DINOv2 features only for crafting perturbations. Using this heuristic, we evaluate the impact of FGSM across the MVTec-AD and VisA datasets and observe consistent drops in F1, AUROC, AP, and G-mean, indicating that imperceptible perturbations can flip nearest-neighbor relations in feature space to induce confident misclassification. Complementing robustness, we probe reliability and find that raw anomaly scores are poorly calibrated, revealing a gap between confidence and correctness that limits safety-critical use. As a simple, strong baseline toward trustworthiness, we apply post-hoc Platt scaling to the anomaly scores for uncertainty estimation. The resulting calibrated posteriors yield significantly higher predictive entropy on adversarially perturbed inputs than on clean ones, enabling a practical flagging mechanism for attack detection while reducing calibration error (ECE). Our findings surface concrete vulnerabilities in DINOv2-based few-shot anomaly detectors and establish an evaluation protocol and baseline for robust, uncertainty-aware anomaly detection. We argue that adversarial robustness and principled uncertainty quantification are not optional add-ons but essential capabilities if anomaly detection systems are to be trustworthy and ready for real-world deployment.
BibTeX
@misc{khan2025towards,
  author = {Akib Mohammed Khan and Bartosz Krawczyk},
  title = {{Towards Adversarial Robustness and Uncertainty Quantification in DINOv2-based Few-Shot Anomaly Detection}},
  year = {2025},
  eprint = {2510.13643},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2510.13643},
}
Figure