Spear: Evaluate the Adversarial Robustness of Compressed Neural Models

Chong Yu, Tao Chen, Zhongxue Gan, Jiayuan FanIJCAI 2024 · OpenReview

Abstract

As Artificial Intelligence evolves, the neural models vulnerable to adversarial attacks may produce fatal results in critical applications. This paper mainly discusses the robustness of the compressed neural models facing adversarial attacks. A few studies discuss the interaction between model compression and adversarial attack. However, they focus on the robustness against the traditional attacks designed for the dense models, not the attacks intended explicitly for the compressed models, using sparsity and quantization techniques. Compressed models often have fewer parameters and smaller sizes that are more friendly to resource-limited devices than dense models, so they are widely deployed in various edge and mobile devices. However, introducing the sparsity and quantization into neural models further imposes higher attack risks. A specific adversarial attack method (Spear) is proposed to generate the particular adversarial attack samples for evaluating the robustness of the compressed models. The Spear attack finds minimal perturbations to create the attack samples to maximize the different behaviors between the compressed and dense reference models. We demonstrate the proposed Spear attack technique can generally be applied to various networks and tasks through quantitative and ablation experiments.

Introduction

To deploy AI on edge devices like smartphones or autonomous vehicles, researchers use model compression—specifically sparsity and quantization—to reduce model size and power consumption. Typically, a compressed model is deemed successful if its accuracy on benign data matches that of the original "dense" model. However, this assumption overlooks the unique security vulnerabilities introduced by compression.

Traditional adversarial attacks (e.g., PGD or FGSM) are designed for dense models and often fail to trigger compression-specific glitches. This creates a false sense of security: a compressed model might appear robust under standard testing while remaining highly vulnerable to an attacker who understands its underlying compression patterns.

Methodology: The Spear Framework

The authors introduce Spear (Specific Attack for Compressed Models), a unified framework that treats sparsity and quantization as specific mathematical transformations of a dense reference model. Unlike traditional attacks that aim to degrade both models similarly, Spear specifically targets the divergence between the dense and compressed versions.

The Spear attack framework maximizes the difference in predictions between dense and compressed models, unlike traditional attacks.

Key components of the Spear framework include:

  • Joint Optimization: Spear finds a perturbation that maximizes the error of the compressed model while minimizing the impact on the dense reference model. This ensures the attack is "stealthy" relative to the original model's behavior.
  • Knowledge Distillation (KD) Loss: By utilizing "soft predictions" (probability distributions), Spear ensures the dense model remains unaffected while the compressed model's logic is pushed toward a misclassification.
  • Attacker & Detector Sub-networks: These ensure that perturbations remain minimal and imperceptible to human observers.

Experimental Evaluation

The researchers evaluated Spear across various tasks, including image classification (ImageNet) and object detection/segmentation (COCO), using architectures like ResNet-50, Swin-V2, RetinaNet, and Mask R-CNN. They targeted multiple compression types, including fine-grained pruning (90% sparse) and various quantization levels (INT8, INT4, FP8).

Key Insights and Results

Spear consistently outperformed state-of-the-art attacks like PGD and Carlini-Wagner in reducing the "Robust Accuracy" of compressed models. A critical finding is that while a compressed model may match a dense model's Top-1 accuracy, its internal logic and "attention" can be vastly different.

Visual comparison of model behavior under Spear attack: CAM visualizations show attention shifts in classification (top), and detection errors (bottom) where a person is misidentified as a snowboard.

As shown in the visualizations above, Class Activation Mapping (CAM) reveals that Spear can shift a model's focus from a primary object to irrelevant background textures. In object detection scenarios, this can lead to catastrophic failures, such as a Mask R-CNN misidentifying a person as a "snowboard"—a mistake that could be fatal in autonomous driving contexts.

Conclusion

Spear demonstrates that model compression is not a neutral transformation regarding security. Pruning and quantization create specific "holes" in a network's logic that can be exploited by targeted attacks. While Spear currently operates in a white-box setting, it serves as a vital evaluation tool. Future work involves using Spear-generated samples for adversarial training to develop compressed models that are truly robust in high-stakes environments.

Summary updated

Abstract

As Artificial Intelligence evolves, the neural models vulnerable to adversarial attacks may produce fatal results in critical applications. This paper mainly discusses the robustness of the compressed neural models facing adversarial attacks. A few studies discuss the interaction between model compression and adversarial attack. However, they focus on the robustness against the traditional attacks designed for the dense models, not the attacks intended explicitly for the compressed models, using sparsity and quantization techniques. Compressed models often have fewer parameters and smaller sizes that are more friendly to resource-limited devices than dense models, so they are widely deployed in various edge and mobile devices. However, introducing the sparsity and quantization into neural models further imposes higher attack risks. A specific adversarial attack method (Spear) is proposed to generate the particular adversarial attack samples for evaluating the robustness of the compressed models. The Spear attack finds minimal perturbations to create the attack samples to maximize the different behaviors between the compressed and dense reference models. We demonstrate the proposed Spear attack technique can generally be applied to various networks and tasks through quantitative and ablation experiments.
BibTeX
@inproceedings{yu2024spear,
  author = {Chong Yu and Tao Chen and Zhongxue Gan and Jiayuan Fan},
  title = {{Spear: Evaluate the Adversarial Robustness of Compressed Neural Models}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2024},
  url = {https://openreview.net/forum?id=URth7zXxRi},
}
Figure