QuantAttack: Exploiting Quantization Techniques to Attack Vision Transformers

Amit Baras, Alon Zolfi, Yuval Elovici, Asaf ShabtaiWACV 2025 · DOI

Abstract

QuantAttack is a novel adversarial attack targeting the "availability" of Vision Transformers (ViTs) by exploiting dynamic quantization. By forcing hardware to utilize high-precision calculations for nearly all operations, QuantAttack significantly increases latency, memory usage, and energy consumption while remaining stealthy to end-users.

Introduction

As Vision Transformers (ViTs) scale, their deployment on edge devices like smartphones and cameras necessitates efficiency optimizations. Quantization is a primary technique used to reduce the computational footprint by representing weights and activations with lower bit-widths (e.g., 8-bit integers). However, dynamic quantization methods like LLM.int8() introduce a critical vulnerability: they preserve accuracy by processing "outliers"—values with unusually large magnitudes—in high-precision (16-bit float). QuantAttack exploits this mechanism to induce a Denial-of-Service (DoS) by artificially creating outliers.

Methodology

QuantAttack utilizes a modified Projected Gradient Descent (PGD) approach to generate adversarial perturbations. The attack optimizes a dual-objective loss function:

  1. Quantization Loss ($\mathcal{L}_{quant}$): This objective identifies the top-$K$ values in each column of the input matrix and pushes them toward a target threshold ($\tau$) to trigger high-precision processing.
  2. Classification Loss ($\mathcal{L}_{cls}$): This ensures the model’s final prediction remains unchanged, making the attack harder to detect as the model continues to function correctly while consuming excessive resources.

The attack was evaluated in single-image, class-universal, and universal variants across multiple domains, including image classification (ImageNet), object detection (OWLv2, YOLOS), and speech recognition (Whisper).

Key Insights: The LayerNorm Vulnerability

A critical finding is the role of Layer Normalization (LayerNorm) in Transformers. The learnable scale parameter ($\gamma$) in later blocks often amplifies values, making it easier for an attacker to generate outliers that exceed quantization thresholds. This architectural feature acts as a natural amplifier for the QuantAttack perturbations.

Experimental Results

QuantAttack effectively degrades hardware performance across various models. For instance, in single-image attacks on ViT, the number of outliers increased by 1681%, leading to significant spikes in resource consumption, such as a 17.2% increase in GPU memory usage and a 9% increase in processing time.

The impact of QuantAttack on memory usage across different batch sizes.

A key observation is the relationship between batch size and attack effectiveness. As shown in the experimental data, the relative overhead in memory and processing time decreases as batch sizes increase. This occurs because larger batches naturally contain more outliers, reducing the marginal impact of synthetic adversarial perturbations.

The impact of QuantAttack on processing time across different batch sizes.

The attack also demonstrated cross-domain success, increasing energy consumption in audio models like Whisper and detection models like OWLv2, confirming that the vulnerability is inherent to the transformer architecture and dynamic quantization, rather than being limited to image classification.

Conclusion and Future Work

QuantAttack highlights a shift from integrity-based attacks to availability-based attacks. While countermeasures like limiting high-precision operations exist, they often trade off model accuracy on legitimate data. Future research will explore the impact of such attacks on Large Language Models (LLMs), where dynamic quantization is increasingly prevalent due to the massive scale of the models.

Summary updated

BibTeX
@inproceedings{baras2025quantattack,
  author = {Amit Baras and Alon Zolfi and Yuval Elovici and Asaf Shabtai},
  title = {{QuantAttack: Exploiting Quantization Techniques to Attack Vision Transformers}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2025},
  doi = {10.1109/wacv61041.2025.00655},
}
Figure