Abstract
Neural networks are known to be vulnerable to adversarial examples: inputs that are close to natural inputs but classified incorrectly. In order to better understand the space of adversarial examples, we survey ten recent proposals that are designed for detection and compare their efficacy. We show that all can be defeated by constructing new loss functions. We conclude that adversarial examples are significantly harder to detect than previously appreciated, and the properties believed to be intrinsic to adversarial examples are in fact not. Finally, we propose several simple guidelines for evaluating future proposed defenses.
Introduction
Deep neural networks are notoriously fragile. A tiny, often imperceptible perturbation to an image—an "adversarial example"—can cause a state-of-the-art model to misclassify a stop sign as a speed limit sign. As achieving perfect robustness has proven elusive, the research community shifted focus toward detection. The goal of detection is not to classify an adversarial image correctly, but to build a "smoke detector" that flags and rejects suspicious inputs.
In this work, we investigate whether these detectors are truly robust or if they merely provide a false sense of security. We demonstrate that an adaptive attacker, who is aware of the detection mechanism, can craft "evasive" adversarial examples that simultaneously fool both the primary classifier and the detector.
Detection Strategies and Threat Models
We evaluated ten different detection schemes, which generally fall into three categories:
- Secondary Classifiers: Training a separate neural network to distinguish between natural and adversarial inputs.
- Principal Component Analysis (PCA): Utilizing statistical techniques to identify abnormalities in the distribution of images or their internal representations.
- Distributional and Randomization Tests: Employing methods like Kernel Density Estimation (KDE) or Dropout randomization to measure prediction uncertainty.
To rigorously test these methods, we considered three threat models: Zero-Knowledge (attacker is unaware of the detector), Perfect-Knowledge (white-box access to parameters), and Limited-Knowledge (black-box access relying on transferability).
Key Insights: The MNIST Trap and Feature Distributions
A major finding of our study is the "MNIST Trap." Many defenses appeared successful on the MNIST dataset only because of its unique properties, such as pure black borders. Adversarial attacks on MNIST often "smear" pixels into these borders, creating artifacts that PCA easily detects. However, on more complex datasets like CIFAR-10, these artifacts do not exist, and the defenses fail.
Furthermore, we challenge the assumption that adversarial examples possess "intrinsic" properties that distinguish them from natural data. While standard adversarial examples might appear as outliers in certain feature spaces (as seen in the PCA component analysis above), an adaptive attacker can move these examples back into the "natural" distribution of the model's internal layers without sacrificing their ability to flip the label.
Results: Systematic Failure of Detection
Our evaluation yielded a sobering result: all ten detection methods were bypassed in the white-box setting. By crafting custom loss functions that account for the detector's logic, we generated adversarial examples that were visually indistinguishable from the originals yet completely invisible to the detectors.
Key findings include:
- Minimal Distortion Cost: For most defenses, the attacker only needed to increase image distortion by approximately 10% to bypass the detector.
- CIFAR-10 Vulnerability: Defenses with high accuracy on MNIST often dropped to near-zero effectiveness on CIFAR-10.
- Transferability Success: Even in black-box scenarios, attacks were highly effective. For instance, 98% of adversarial examples generated against a substitute model successfully fooled the Dropout-based detector on MNIST.
Conclusion and Recommendations
While we have shown that these ten specific methods are vulnerable, we do not claim that all detection is impossible. However, our results suggest that adversarial examples are significantly harder to detect than previously believed. To improve the rigor of future research, we propose that future evaluations must include mandatory white-box testing, move beyond MNIST-only evaluations, and report full ROC curves to properly characterize the trade-off between false positives and detection accuracy.
Summary updated