Inference Time Evidences of Adversarial Attacks for Forensic on Transformers

Hugo Lemarchant, Liangzi Li, Yiming Qian, Yuta Nakashima, Hajime NagaharaarXiv 2023 · DOI · arXiv

Introduction

Vision Transformers (ViTs) have demonstrated state-of-the-art performance across various computer vision tasks. However, like their convolutional counterparts, they remain susceptible to adversarial attacks—carefully crafted, imperceptible perturbations that lead to incorrect classifications. While traditional research focuses on adversarial training to enhance robustness, this study explores a "forensic" approach: identifying the unique signatures left by adversarial inputs during the inference process.

Forensic Quantifications

The researchers propose four distinct metrics to monitor the internal and external behavior of ViTs during inference:

  1. Frequency Ratio (FR): Analyzes the input image's energy distribution using Discrete Cosine Transform (DCT). Adversarial noise typically alters the balance between high and low-frequency components.
  2. Posterior Entropy (PH): Measures the uncertainty in the model's final probability distribution. Attacks often induce higher entropy as the model becomes "confused" between classes.
  3. Attention Distance (AD): Tracks the physical distance between image patches connected by the transformer's attention mechanism. Attacks can force the model to attend to irrelevant, distant regions.
  4. CKA Similarity (SCKA): Uses Centered Kernel Alignment to compare internal layer representations between clean and potentially attacked samples, detecting disruptions in internal logic.

Detection Performance

Experimental results on ImageNet-1K reveal that the most effective "alarms" for detecting attacks are the input and output signals.

Distribution of Input Frequency Ratio across different attack types for ViT-S/16 and T2T-ViT-t models.

As shown in Figure 2, attacks like FGSM significantly shift the Frequency Ratio distribution, making them relatively easy to distinguish from clean samples. Similarly, the Posterior Entropy (Figure 3) provides a clear signal of adversarial interference, particularly for models like T2T-ViT-t which exhibit more varied entropy profiles under attack.

Distribution of Output Entropy across different attack types, highlighting the shift in model uncertainty.

The Challenge of Stealth Attacks

While rougher attacks like FGSM and PGD leave detectable traces, highly optimized attacks such as C&W (Carlini & Wagner) present a significant challenge. For C&W attacks, the Bhattacharyya Coefficient (a measure of distribution overlap) remained near 1.00 across almost all metrics, indicating that the attack is forensically invisible to these specific quantifications. This suggests that attackers with sufficient computational budget can bypass both the classifier and the forensic monitors.

Conclusion and Future Directions

The study highlights that while internal latent features (like Attention Distance) are excellent for explaining how an attack disrupts a model, they are less effective as primary detection triggers due to high overlap with clean data distributions. Future work aims to move beyond single-metric thresholds toward multi-dimensional forensic profiles that combine frequency, attention, and entropy data to catch even the most sophisticated stealth attacks. Additionally, identifying "volatile heads" in early transformer blocks may lead to more targeted hardening strategies.

Summary updated

Abstract

Vision Transformers (ViTs) are becoming a very popular paradigm for vision tasks as they achieve state-of-the-art performance on image classification. However, although early works implied that this network structure had increased robustness against adversarial attacks, some works argue ViTs are still vulnerable. This paper presents our first attempt toward detecting adversarial attacks during inference time using the network's input and outputs as well as latent features. We design four quantifications (or derivatives) of input, output, and latent vectors of ViT-based models that provide a signature of the inference, which could be beneficial for the attack detection, and empirically study their behavior over clean samples and adversarial samples. The results demonstrate that the quantifications from input (images) and output (posterior probabilities) are promising for distinguishing clean and adversarial samples, while latent vectors offer less discriminative power, though they give some insights on how adversarial perturbations work.
BibTeX
@misc{lemarchant2023inference,
  author = {Hugo Lemarchant and Liangzi Li and Yiming Qian and Yuta Nakashima and Hajime Nagahara},
  title = {{Inference Time Evidences of Adversarial Attacks for Forensic on Transformers}},
  year = {2023},
  eprint = {2301.13356},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2301.13356},
}
Figure