On the reversibility of adversarial attacks

Chau Yi Li, Ricardo Sánchez-Matilla, Ali Shahin Shamsabadi, Riccardo Mazzon, Andrea Cavallaro

Abstract

Adversarial attacks modify images with perturbations that change the prediction of classifiers. These modified images, known as adversarial examples, expose the vulnerabilities of deep neural network classifiers. In this paper, we investigate the predictability of the mapping between the classes predicted for original images and for their corresponding adversarial examples. This predictability relates to the possibility of retrieving the original predictions and hence reversing the induced misclassification. We refer to this property as the reversibility of an adversarial attack, and quantify reversibility as the accuracy in retrieving the original class or the true class of an adversarial example. We present an approach that reverses the effect of an adversarial attack on a classifier using a prior set of classification results. We analyse the reversibility of state-of-the-art adversarial attacks on benchmark classifiers and discuss the factors that affect the reversibility.

Introduction

Adversarial attacks are often viewed as chaotic disruptions that break model logic, but this paper reveals they are surprisingly predictable. Deep Neural Networks (DNNs) are notoriously vulnerable to adversarial attacks—tiny, often imperceptible perturbations to an image that cause a model to misclassify it. While most research focuses on making models more robust or creating stronger attacks, this paper asks a different question: Is the damage reversible?

If a model misclassifies an adversarial "stop sign" as a "speed limit sign," is that specific error random, or is there a hidden pattern? The authors argue that "reversibility" is a fundamental property of an attack. If we can understand the mapping between the original class and the adversarial class, we can potentially retrieve the true identity of the image even after it has been tampered with.

Methodology: Prediction Reversal from the Adversarial Class (PRAC)

The authors propose a framework called PRAC (Prediction Reversal from the Adversarial Class). Instead of trying to "clean" the image (a common defense), PRAC treats the adversarial classification as a statistical clue.

  1. Mapping Discovery: The researchers first build a "weighted bipartite graph" by attacking a known dataset. They observe how often class $A$ gets turned into class $B$.
  2. Probability Estimation: PRAC calculates the probability that an adversarial class $y_j$ originated from a specific original class $y_k$.
  3. Soft-Output Integration: Unlike a simple frequency count, PRAC leverages the "soft" outputs (the probability distribution) of the classifier’s last layer. This allows the system to account for the model's internal uncertainty, which proves more accurate than just looking at the final "hard" label.
  4. Retrieval: When a new, unknown adversarial image appears, PRAC looks at the predicted adversarial class and uses its pre-calculated probabilities to suggest the most likely original classes (Top-1 or Top-5).
Bipartite graph illustrating the mapping between true, original, and adversarial classes.

Experimental Setup

The study evaluates reversibility across two distinct scales:

  • MNIST: The classic dataset of handwritten digits (10 classes). This served as a baseline using the LeNet classifier.
  • Places365-Standard: A much more complex scene recognition dataset with 365 classes. This was used to test the approach on modern architectures like AlexNet, ResNet-18, and ResNet-50.

Key Insights: Preferential Mapping

A core finding is that adversarial attacks are not "random noise." * Preferential Mapping: Non-targeted attacks (like DeepFool) tend to push an image to the second most likely class in the original prediction. Targeted attacks (like L-FGSM) create even stronger, predictable paths between specific classes. * Reversibility is Attack-Centric: Interestingly, the reversibility of an attack remains fairly consistent regardless of the specific DNN architecture (AlexNet vs. ResNet). It is a property of the attack algorithm itself. * The Confidence Trade-off: If an attacker forces the model to be highly confident in its wrong prediction (using a high threshold $\sigma$), the attack becomes less reversible. High-confidence attacks break the natural "preferential mapping" that PRAC exploits. * DeepFool is Highly Reversible: Because DeepFool moves images to the nearest decision boundary, it almost always lands on a class that was already "close" in the model's original logic, making it very easy to trace back.

Mapping density for the DeepFool attack, showing structured preferential transitions.

Quantitative Results

The results demonstrate that "reversing" an attack is a viable defense strategy:

  • MNIST Recovery: On the MNIST dataset, an N-FGSM attack reduced model accuracy to a measly 1.40%. However, using PRAC, the researchers could retrieve the true class with 39.80% accuracy.
  • Places365 Performance: For the complex ResNet-50 model, PRAC achieved a Top-5 retrieval accuracy of 36.48% for DeepFool attacks.
  • Predictability of Errors: In non-targeted attacks on Places365, the adversarial class was one of the original top 2-5 most likely classes in over 92% of cases for DeepFool. This confirms that attacks usually "nudge" the model to its next best guess rather than a random one.

Limitations and Conclusion

While PRAC is a powerful diagnostic tool, it has some hurdles. It requires a "prior set" of attacked images to build its mapping. If an attacker uses a brand-new, unseen attack method, the mapping might not hold. Additionally, future adversarial attacks could be designed to be "irreversible" by intentionally choosing target classes that have no statistical relationship to the source class.

In summary, this paper shifts the perspective on adversarial examples from "unrecoverable errors" to "predictable transformations," opening a new front in the cat-and-mouse game of AI security.

Summary updated

Abstract

Adversarial attacks modify images with perturbations that change the prediction of classifiers. These modified images, known as adversarial examples, expose the vulnerabilities of deep neural network classifiers. In this paper, we investigate the predictability of the mapping between the classes predicted for original images and for their corresponding adversarial examples. This predictability relates to the possibility of retrieving the original predictions and hence reversing the induced misclassification. We refer to this property as the reversibility of an adversarial attack, and quantify reversibility as the accuracy in retrieving the original class or the true class of an adversarial example. We present an approach that reverses the effect of an adversarial attack on a classifier using a prior set of classification results. We analyse the reversibility of state-of-the-art adversarial attacks on benchmark classifiers and discuss the factors that affect the reversibility.
BibTeX
@inproceedings{li2021reversibility,
  author = {Chau Yi Li and Ricardo Sánchez-Matilla and Ali Shahin Shamsabadi and Riccardo Mazzon and Andrea Cavallaro},
  title = {{On the reversibility of adversarial attacks}},
  booktitle = {ICIP},
  year = {2021},
  url = {https://openreview.net/forum?id=AvYQpAQvK4W},
}
Figure