Meta-Attack: Class-agnostic and Model-agnostic Physical Adversarial Attack

Weiwei Feng, Baoyuan Wu, Tianzhu Zhang, Yong Zhang 0034, Yongdong Zhang 0001

Abstract

Physical adversarial attacks, such as stickers designed to deceive autonomous vehicle sensors, often struggle with environmental distortions and lack the ability to generalize across different object classes or AI models. We introduce Meta-Attack, a framework that reformulates physical attack generation as a few-shot learning problem. By meta-training a generative model on a diverse set of attack tasks, the system can rapidly adapt to unseen objects and architectures using only a handful of physical images. Experimental results on the GTSRB and ImageNet datasets demonstrate that Meta-Attack significantly outperforms traditional optimization-based methods, achieving a 95.2% success rate in physical environments and maintaining high effectiveness across varying camera angles.

Introduction

Adversarial attacks—subtle image perturbations that trigger misclassification in deep neural networks (DNNs)—are a well-known security risk. However, deploying these attacks in the physical world is challenging due to the "Digital-to-Physical" (D2P) transformation. Factors such as lighting, camera perspective, and printing textures introduce distortions that typically neutralize digital adversarial noise.

Current state-of-the-art physical attacks often rely on "Expectation Over Transformation" (EOT), which simulates thousands of physical variations during a slow, iterative optimization process. This process must be repeated from scratch for every new target image or victim model, making it computationally expensive and inflexible.

Comparison of physical attack paradigms: (a) standard iterative optimization, (b) large-scale single-sample attack, and (c) the proposed multi-class, multi-model Meta-Attack.

Methodology

Meta-Attack moves away from per-image optimization toward a generalized generative approach. The framework is built on two pillars:

1. Generative Attack Model

Instead of directly optimizing pixels, Meta-Attack employs a generative model (based on CycleGAN) to produce adversarial perturbations. This generator is trained to account for the color and shape distortions inherent in the D2P process, ensuring that the generated noise remains robust when printed and photographed.

2. Class-agnostic and Model-agnostic Meta-Learning (CMML)

The core innovation is treating the attack as a few-shot learning task. Using a meta-learning algorithm, the generator is trained across a wide variety of "attack tasks" involving different object classes and DNN architectures.

  • Meta-Training: The model learns a generalized parameter initialization by attacking various known classes and models.
  • Meta-Testing (Fine-tuning): When encountering a new object or model, the generator uses a small "support set" of physical images to fine-tune its parameters. After minimal gradient descent steps, it can generate effective adversarial versions of the new target.
The Meta-Attack framework: Meta-training across multiple tasks allows the generator to quickly adapt to unseen tasks via a few-shot fine-tuning process.

Experimental Results

The researchers evaluated Meta-Attack using the ImageNet and German Traffic Sign Recognition Benchmark (GTSRB) datasets.

  • Physical Success Rate: On GTSRB, Meta-Attack achieved an Attack Success Rate (ASR) of 95.2% in physical settings, outperforming baselines like EOT and RP2, which typically range between 70-80%.
  • Angular Robustness: The attack proved resilient to perspective changes. While other methods failed at extreme angles, Meta-Attack maintained an ASR above 65% for camera angles ranging from -45° to 45°.
  • Cross-Model Generalization: The framework demonstrated strong "black-box" performance. A generator meta-trained on one architecture (e.g., VGG-16) could be successfully fine-tuned to attack another (e.g., ResNet-50) with high efficiency.

Conclusion and Future Work

Meta-Attack represents a significant step toward making physical adversarial attacks more practical and scalable. By leveraging meta-learning, it overcomes the need for exhaustive per-target optimization.

However, limitations remain. The requirement for even a "few" physical samples for fine-tuning suggests that future research should aim for "zero-shot" physical attacks. Additionally, improving the imperceptibility of these physical patches—making them look like natural environmental artifacts—is a key area for future development. Finally, exploring how adversarial training can be adapted to defend against meta-learned attacks is crucial for securing future AI systems.

Summary updated

Abstract

Modern deep neural networks are often vulnerable to adversarial examples. Most exist attack methods focus on crafting adversarial examples in the digital domain, while only limited works study physical adversarial attack. However, it is more challenging to generate effective adversarial examples in the physical world due to many uncontrollable physical dynamics. Most current physical attack methods aim to generate robust physical adversarial examples by simulating all possible physical dynamics. When attacking new images or new DNN models, they require expensive manually efforts for simulating physical dynamics and considerable time for iteratively optimizing for each image. To tackle these issues, we propose a class-agnostic and model-agnostic physical adversarial attack model (Meta-Attack), which is able to not only generate robust physical adversarial examples by simulating color and shape distortions, but also generalize to attacking novel images and novel DNN models by accessing a few digital and physical images. To the best of our knowledge, this is the first work to formulate the physical attack as a few-shot learning problem. Here, the training task is redefined as the composition of a support set, a query set, and a target DNN model. Under the few- shot setting, we design a novel class-agnostic and model-agnostic meta-learning algorithm to enhance the generalization ability of our method. Extensive experimental results on two benchmark datasets with four challenging experimental settings verify the superior robustness and generalization of our method by comparing to state-of-the-art physical attack methods.
BibTeX
@inproceedings{feng2021meta,
  author = {Weiwei Feng and Baoyuan Wu and Tianzhu Zhang and Yong Zhang and Yongdong Zhang},
  title = {{Meta-Attack: Class-agnostic and Model-agnostic Physical Adversarial Attack}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2021},
  doi = {10.1109/iccv48922.2021.00769},
}
Figure