Mimic and Fool: A Task Agnostic Adversarial Attack

Akshay Chaturvedi, Utpal GarainIEEE TNNLS 2019 · DOI · OpenReview

Abstract

At present, adversarial attacks are designed in a task-specific fashion. However, for downstream computer vision tasks such as image captioning and image segmentation, current deep-learning systems use an image classifier such as VGG16, ResNet50, or Inception-v3 as a feature extractor. Keeping this in mind, we propose Mimic and Fool (MaF), a task-agnostic adversarial attack. Given a feature extractor, the proposed attack finds an adversarial image that mimics the image features of the original image. This ensures that the two images produce similar outputs regardless of the specific downstream task. We evaluate MaF on image captioning and visual question answering (VQA) models, achieving success rates up to 87.1%. Since MaF only requires information about the feature extractor, it serves as an effective gray-box attack.

Introduction

Most existing adversarial attacks are "task-specific," meaning they are optimized to fool a particular output, such as a classification label or a specific sentence in image captioning. However, modern computer vision pipelines are modular, typically employing a "backbone" feature extractor (e.g., a CNN) to convert images into mathematical vectors before passing them to a task-specific "head."

This research identifies a systemic vulnerability in this modularity. If an attacker can manipulate an image to produce a specific feature vector at the backbone level, they can control the output of any downstream task using that backbone. This makes the attack "task-agnostic," allowing a single adversarial image to fool multiple different models simultaneously.

Methodology

The researchers propose two primary variants of the attack:

  1. Mimic and Fool (MaF): This gray-box attack targets a known feature extractor. Starting from a blank (zero) image, the attack uses optimization to find an adversarial image ($I_{adv}$) such that its feature vector $f(I_{adv})$ closely matches that of a target natural image ($I_{org}$). Because CNNs are non-invertible, the optimization often results in a noisy image that appears as random static to humans but is perceived as the target image by the model.
A sample natural image from the MSCOCO dataset used as a target for feature mimicry.
  1. One Image Many Outputs (OIMO): To increase deceptiveness, OIMO starts with a "base" natural image (e.g., a dog) and adds minimal perturbations so its feature vector mimics a completely different target (e.g., a kitchen). The result is an image that looks like one thing to a human but is processed as another by the AI.
An adversarial image generated using the MaF attack, which appears as noise to humans but mimics the features of a target image for the model.

Experimental Evaluation

The attack was tested using 1,000 MSCOCO validation images and 5,208 VQA v2.0 image-question pairs across three major models: Show and Tell (Captioning), Show Attend and Tell (Captioning), and N2NMN (VQA).

  • Success Rates: MaF achieved success rates of 74.0% for Show and Tell, 81.0% for Show Attend and Tell, and 87.1% for N2NMN.
  • Efficiency: OIMO is remarkably fast, generating an attack in under 8 seconds per image, significantly outperforming task-specific methods like Show-and-Fool, which can take nearly 178 seconds.
  • Invertible Architectures: The authors also successfully attacked i-RevNet, proving that even mathematically invertible architectures are susceptible if the features assigned to dissimilar images remain close in vector space.

Discussion and Key Insights

The success of MaF suggests that CNN feature extractors are "chaotic"; slight pixel perturbations can map almost any image to the representation of another. By "hacking the eyes" of the model—the feature extractor—the attacker bypasses the need to understand the complex "brain" (the task-specific head).

While task-agnostic attacks currently introduce more noise than task-specific ones, the OIMO method demonstrates that this gap is closing. A limitation remains the gray-box requirement, as the attacker must know which backbone the target system uses. Future work should focus on developing feature extractors that are explicitly trained to resist feature-mimicry, potentially using the MaF framework for adversarial training.

Conclusion

Mimic and Fool demonstrates that the modular nature of modern AI is a double-edged sword. While shared backbones allow for efficient transfer learning, they also create a single point of failure. This research highlights the urgent need for robust feature extraction layers that can distinguish between natural signals and adversarial mimicry.

Summary updated

Abstract

At present, adversarial attacks are designed in a task-specific fashion. However, for downstream computer vision tasks such as image captioning, image segmentation etc., the current deep learning systems use an image classifier like VGG16, ResNet50, Inception-v3 etc. as a feature extractor. Keeping this in mind, we propose Mimic and Fool, a task agnostic adversarial attack. Given a feature extractor, the proposed attack finds an adversarial image which can mimic the image feature of the original image. This ensures that the two images give the same (or similar) output regardless of the task. We randomly select 1000 MSCOCO validation images for experimentation. We perform experiments on two image captioning models, Show and Tell, Show Attend and Tell and one VQA model, namely, end-to-end neural module network (N2NMN). The proposed attack achieves success rate of 74.0%, 81.0% and 87.1% for Show and Tell, Show Attend and Tell and N2NMN respectively. We also propose a slight modification to our attack to generate natural-looking adversarial images. In addition, we also show the applicability of the proposed attack for invertible architecture. Since Mimic and Fool only requires information about the feature extractor of the model, it can be considered as a gray-box attack.
BibTeX
@article{chaturvedi2019mimic,
  author = {Akshay Chaturvedi and Utpal Garain},
  title = {{Mimic and Fool: A Task Agnostic Adversarial Attack}},
  journal = {IEEE Transactions on Neural Networks and Learning Systems},
  year = {2019},
  doi = {10.1109/tnnls.2020.2984972},
}
Figure