Adversarial Examples as an Input-Fault Tolerance Problem

Angus Galloway, Anna Golubeva, Graham W. TaylorarXiv 2018 · OpenReview

TL;DR

This paper reframes adversarial examples not as a unique attack, but as a manifestation of a model's input-fault tolerance. They propose using mutual information between model predictions and true labels as a more robust metric for evaluating this tolerance than simple accuracy, especially under diverse, realistic input perturbations.

What Problem Does This Paper Address?

The field of adversarial machine learning has largely focused on crafting specific, often mathematically constrained, perturbations (like $\epsilon$-balls) to fool neural networks. While these studies reveal vulnerabilities, they don't fully capture how a model behaves under a broader range of real-world input "faults" – deviations from clean data that can arise from noise, sensor errors, or even natural variations. This paper argues that adversarial examples are a symptom of poor input-fault tolerance, a concept more traditionally applied to internal system failures. The challenge is to develop a principled way to measure and understand this input-fault tolerance across a spectrum of realistic input perturbations, moving beyond the limitations of fixed threat models.

Methods

The core contribution is a new metric for evaluating input-fault tolerance: the mutual information (MI) between the model's categorical prediction ($T$) and the true label ($Y$), denoted as $I(T; Y)$. This metric quantifies how much information about the true label is preserved in the model's output, even when the input is corrupted.

  • Information-Theoretic Approach: Instead of just looking at prediction accuracy (which can be misleading, as shown in the paper), they track how $I(T; Y)$ changes as the strength of input perturbations increases.
  • Diverse Fault Models: They evaluate this metric against various types of input faults, including:
  • Additive White Gaussian Noise (AWGN)
    • Rotations and translations
    • Basic Iterative Method (BIM) attacks (both L2 and L∞ variants)
    • "Rubbish" or "fooling" images
    • Vector field-based deformations ("ADef")
  • Model Training: They compare models trained with and without weight decay, demonstrating how regularization can improve fault tolerance.

Datasets

  • SVHN (Street View House Numbers): Used for training and evaluation. The authors note they use a random sample of 1000 images from the test set for their analysis.
  • No new benchmarks introduced.

Key Insights and Ideas

  • Adversarial Examples as Input Faults: The paper successfully reframes adversarial examples as a specific type of input fault, broadening the perspective beyond targeted attacks.
  • Mutual Information as a Superior Metric: $I(T; Y)$ provides a richer picture of model behavior than accuracy. For instance, a model can have near-zero accuracy but still convey meaningful (though incorrect) information about the input's intended class, which MI captures. Conversely, MI can degrade gracefully even when accuracy plummets.
  • Weight Decay Enhances Fault Tolerance: Models trained with weight decay exhibit more gradual degradation of $I(T; Y)$ under attack, suggesting improved robustness and more interpretable adversarial examples (e.g., cleaner "fooling images").
  • Targeted Attacks Require More Distortion: The paper shows that to achieve a specific performance drop, targeted adversarial attacks generally require more significant input perturbations compared to simple misclassification attacks.
  • Predictable Degradation is Key: The "aha moment" is that a model's fault tolerance is characterized not just by how much its performance degrades, but how predictably it degrades. Gradual, consistent degradation of $I(T; Y)$ is a sign of better robustness.

Key Results

  • Figure 1: Demonstrates that $I(T; Y)$ curves for a model trained with weight decay degrade more gradually and predictably than for a model without weight decay, under various attacks. Accuracy curves, while also degrading, don't offer the same nuanced insight.
  • Figure 2: Visualizes "fooling images" generated by a BIM-L2 attack. Images generated for the weight-decayed model are cleaner and more clearly resemble the target digit, whereas the baseline model's images are noisy and less interpretable.
  • Figure 3: Shows that spatial attacks (rotations, deformations) also benefit from weight decay, with more gradual MI degradation.
  • Figure 5: Compares L2 and L∞ BIM attacks, showing L∞ requires more distortion for a given performance drop, as expected.
  • Figure 6 & 7: Provide further visual examples of adversarial examples and deformations, reinforcing the idea that weight decay leads to more interpretable and less noisy adversarial perturbations.

Limitations and Future Work

  • Test Set Attack: The authors acknowledge their SVHN solution's accuracy (90.4%) is not state-of-the-art (98.7%), suggesting future work could focus on closing this gap.
  • Generalizability: While the proposed MI metric is general, the specific findings are based on a particular model architecture and dataset. Further validation on diverse architectures and datasets would be beneficial.
  • Beyond $\epsilon$-Perturbations: The paper advocates for considering a wider range of "valid inputs" beyond strict $\epsilon$-balls. Future work could explore even more complex and realistic fault models.
  • Interpretable Deformations: The qualitative examples of "ADef" attacks are promising, suggesting future work could delve deeper into generating and understanding these types of deformations.

Summary updated

Abstract

We analyze the adversarial examples problem in terms of a model's fault tolerance with respect to its input. Whereas previous work focuses on arbitrarily strict threat models, i.e., $\epsilon$-perturbations, we consider arbitrary valid inputs and propose an information-based characteristic for evaluating tolerance to diverse input faults.
BibTeX
@misc{galloway2018adversarial,
  author = {Angus Galloway and Anna Golubeva and Graham W. Taylor},
  title = {{Adversarial Examples as an Input-Fault Tolerance Problem}},
  year = {2018},
  eprint = {1811.12601},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1811.12601},
}
Figure