PRAT: PRofiling Adversarial aTtacks

Rahul Ambati, Naveed Akhtar, Ajmal Mian, Yogesh Singh RawatWorkshops @ ICCV 2023 · OpenReview

Introduction

Deep learning models are notoriously vulnerable to adversarial attacks—small, often invisible perturbations added to images that cause models to fail. While the research community has spent years developing defenses like adversarial training, these are often bypassed by new strategies. PRAT (PRofiling Adversarial aTtacks) shifts the perspective from defense to forensics. The authors argue that even though adversarial noise is imperceptible to humans, different algorithms leave behind distinct "fingerprints" or traces.

Visual comparison of different adversarial attack signatures (FGSM, DeepFool, CW, Universal) compared to a clean image.

By identifying which attack was used, security practitioners can profile an attacker’s capabilities, understand their access level (e.g., white-box vs. black-box), and deploy targeted countermeasures outside the neural network itself.

The PRAT Framework

The core of PRAT is a hybrid "Global-LOcal Feature" (GLOF) module designed to extract unique "attack signatures."

GLOF Module and Signature Extraction

Standard CNNs are effective at picking up local textures (local features), while Transformers excel at understanding long-range relationships across an image (global features). The GLOF module uses a dual-arm architecture to capture both. Instead of classifying the adversarial image directly, the network learns to "rectify" the image to reconstruct a clean version. By subtracting this rectified image from the adversarial input, the system isolates the attack signature ($ ilde{ ho}$).

Architecture of the GLOF module, showing the integration of Multi-Head Attention and Convolutional Layers for feature extraction and image reconstruction.

Two-Stage Training

  1. Stage 1 (Signature Extraction): The extractor is trained to minimize the distance between the generated signature and the actual ground-truth perturbation.
  2. Stage 2 (Attack Classification): An attack classifier (based on DenseNet121) is trained to identify the attack label using the extracted signature and the original adversarial image as input.

The Adversarial Identification Dataset (AID)

To support this research, the authors curated the Adversarial Identification Dataset (AID), featuring over 180,000 samples across 13 popular attacks. These include gradient-based (white-box), decision-based (black-box), and universal (image-agnostic) families. Attacks were generated against multiple models (ResNet50, DenseNet121, and InceptionV3) to ensure signatures were not merely artifacts of a single victim architecture.

Key Results and Insights

PRAT achieved 80.14% accuracy in identifying specific attack algorithms and 84.72% accuracy for attack families, significantly outperforming standard models like ResNet152 (67.03%) and Vision Transformers (67.28%).

  • Peculiar Patterns: Even when noise is constrained to be invisible (low $\ell_p$ norm), the mathematical "path" an algorithm takes to find that noise leaves a trace.
  • Hybrid Advantage: The paper found that using only CNNs or only Transformers resulted in lower accuracy. Identifying an attack requires looking at both the fine-grained noise texture and how that noise is distributed across the entire image structure.
  • Universal Attacks are "Loud": Universal Adversarial Perturbations (UAP) are the easiest to identify because they are image-agnostic and form very distinct clusters in feature space.
  • Cross-Model Robustness: The system remained effective even when the attack was generated on a model different from the one used during training, proving that attack signatures are somewhat independent of the victim architecture.

Limitations and Future Work

The "Open Set" problem remains a significant challenge; accuracy drops to 57.2% when facing completely unseen attacks. Additionally, future work must address adaptive attackers who might attempt to "spoof" signatures to mislead profiling systems, as well as extending the framework to physical-world perturbations or environmental image corruptions.

Summary updated

Abstract

Intrinsic susceptibility of deep learning to adversarial examples has led to a plethora of attack techniques with a broad common objective of fooling deep models. However, we find slight compositional differences between the algorithms achieving this objective. These differences leave traces that provide important clues for attacker profiling in real-life scenarios. Inspired by this, we introduce a novel problem of PRofiling Adversarial aTtacks (PRAT). Given an adversarial example, the objective of PRAT is to identify the attack used to generate it. Under this perspective, we can systematically group existing attacks into different families, leading to the sub-problem of attack family identification, which we also study. To enable PRAT analysis, we introduce a large Adversarial Identification Dataset (AID), comprising over 180k adversarial samples generated with 13 popular attacks for image specific/agnostic white/black box setups. We use AID to devise a novel framework for the PRAT objective. Our framework utilizes a Transformer based Global-LOcal Feature (GLOF) module to extract an approximate signature of the adversarial attack, which in turn is used for the identification of the attack. Using AID and our framework, we provide multiple interesting benchmark results for the PRAT problem.
BibTeX
@inproceedings{ambati2023prat,
  author = {Rahul Ambati and Naveed Akhtar and Ajmal Mian and Yogesh Singh Rawat},
  title = {{PRAT: PRofiling Adversarial aTtacks}},
  booktitle = {Workshops @ ICCV},
  year = {2023},
  url = {https://openreview.net/forum?id=R6GHaWwAk2},
}
Figure