On the Reversibility of Adversarial Attacks

Chau Yi Li, Ricardo Sánchez-Matilla, Ali Shahin Shamsabadi, Riccardo Mazzon, Andrea Cavallaro

TL;DR

Adversarial attacks are often viewed as chaotic disruptions that break model logic, but this paper reveals they are surprisingly predictable. By analyzing the "preferential mapping" between original and adversarial classes, the authors introduce PRAC (Prediction Reversal from the Adversarial Class), a method that can effectively "undo" an attack and recover the original label.

What Problem Does This Paper Address?

Deep Neural Networks (DNNs) are notoriously vulnerable to adversarial attacks—tiny, often imperceptible perturbations to an image that cause a model to misclassify it. While most research focuses on making models more robust or creating stronger attacks, this paper asks a different question: Is the damage reversible?

If a model misclassifies an adversarial "stop sign" as a "speed limit sign," is that specific error random, or is there a hidden pattern? The authors argue that "reversibility" is a fundamental property of an attack. If we can understand the mapping between the original class and the adversarial class, we can potentially retrieve the true identity of the image even after it has been tampered with.

Methods

The authors propose a framework called PRAC (Prediction Reversal from the Adversarial Class). Instead of trying to "clean" the image (a common defense), PRAC treats the adversarial classification as a statistical clue.

  1. Mapping Discovery: The researchers first build a "weighted bipartite graph" by attacking a known dataset. They observe how often class $A$ gets turned into class $B$.
  2. Probability Estimation: PRAC calculates the probability that an adversarial class $y_j$ originated from a specific original class $y_k$.
  3. Soft-Output Integration: Unlike a simple frequency count, PRAC leverages the "soft" outputs (the probability distribution) of the classifier’s last layer. This allows the system to account for the model's internal uncertainty, which proves more accurate than just looking at the final "hard" label.
  4. Retrieval: When a new, unknown adversarial image appears, PRAC looks at the predicted adversarial class and uses its pre-calculated probabilities to suggest the most likely original classes (Top-1 or Top-5).

Datasets

The study evaluates reversibility across two distinct scales:

  • MNIST: The classic dataset of handwritten digits (10 classes). This served as a baseline using the LeNet classifier.
  • Places365-Standard: A much more complex scene recognition dataset with 365 classes. This was used to test the approach on modern architectures like AlexNet, ResNet-18, and ResNet-50.

Key Insights and Ideas

  • Preferential Mapping: Adversarial attacks are not "random noise." Non-targeted attacks (like DeepFool) tend to push an image to the second most likely class in the original prediction. Targeted attacks (like L-FGSM) create even stronger, predictable paths between specific classes.
  • Reversibility is Attack-Centric: Interestingly, the reversibility of an attack remains fairly consistent regardless of the specific DNN architecture (AlexNet vs. ResNet). It is a property of the attack algorithm itself.
  • The Confidence Trade-off: The authors found that if an attacker forces the model to be highly confident in its wrong prediction (using a high threshold $\sigma$), the attack becomes less reversible. High-confidence attacks break the natural "preferential mapping" that PRAC exploits.
  • DeepFool is Highly Reversible: Because DeepFool moves images to the nearest decision boundary, it almost always lands on a class that was already "close" in the model's original logic, making it very easy to trace back.

Key Results

The quantitative results demonstrate that "reversing" an attack is a viable defense strategy:

  • MNIST Recovery: On the MNIST dataset, an N-FGSM attack reduced model accuracy to a measly 1.40%. However, using PRAC, the researchers could retrieve the true class with 39.80% accuracy.
  • Places365 Performance: For the complex ResNet-50 model, PRAC achieved a Top-5 retrieval accuracy of 36.48% for DeepFool attacks.
  • Predictability of Errors: In non-targeted attacks on Places365, the adversarial class was one of the original top 2-5 most likely classes in over 92% of cases for DeepFool. This confirms that attacks usually "nudge" the model to its next best guess rather than a random one.

Limitations and Future Work

While PRAC is a powerful diagnostic tool, it has some hurdles:

  • Prior Knowledge Required: PRAC requires a "prior set" of attacked images to build its mapping. If an attacker uses a brand-new, unseen attack method, the mapping might not hold.
  • The "Random" Counter-Attack: The authors suggest that future adversarial attacks could be designed to be "irreversible" by intentionally choosing target classes that have no statistical relationship to the source class (similar to the R-FGSM attack studied in the paper, which showed the lowest reversibility).
  • Scaling to Open-Set: The current work assumes a closed set of classes. Expanding this to "in-the-wild" scenarios where the original class might not be in the training set remains a challenge.

In summary, this paper shifts the perspective on adversarial examples from "unrecoverable errors" to "predictable transformations," opening a new front in the cat-and-mouse game of AI security.

Summary updated

Abstract

Adversarial attacks modify images with perturbations that change the prediction of classifiers. These modified images, known as adversarial examples, expose the vulnerabilities of deep neural network classifiers. In this paper, we investigate the predictability of the mapping between the classes predicted for original images and for their corresponding adversarial examples. This predictability relates to the possibility of retrieving the original predictions and hence reversing the induced misclassification. We refer to this property as the reversibility of an adversarial attack, and quantify reversibility as the accuracy in retrieving the original class or the true class of an adversarial example. We present an approach that reverses the effect of an adversarial attack on a classifier using a prior set of classification results. We analyse the reversibility of state-of-the-art adversarial attacks on benchmark classifiers and discuss the factors that affect the reversibility.
BibTeX
@inproceedings{li2021reversibility,
  author = {Chau Yi Li and Ricardo Sánchez-Matilla and Ali Shahin Shamsabadi and Riccardo Mazzon and Andrea Cavallaro},
  title = {{On the Reversibility of Adversarial Attacks}},
  booktitle = {ICIP},
  year = {2021},
  url = {https://openreview.net/forum?id=AvYQpAQvK4W},
}

Loading related content...

Figure