Abstract
Neural networks provide state-of-the-art results for most machine learning tasks. Unfortunately, neural networks are vulnerable to adversarial examples: given an input $x$ and any target classification $t$, it is possible to find a new input $x'$ that is similar to $x$ but classified as $t$. This makes it difficult to apply neural networks in security-critical areas. Defensive distillation is a recently proposed approach that can take an arbitrary neural network, and increase its robustness, reducing the success rate of current attacks' ability to find adversarial examples from $95\%$ to $0.5\%$. In this paper, we demonstrate that defensive distillation does not significantly increase the robustness of neural networks by introducing three new attack algorithms that are successful on both distilled and undistilled neural networks with $100\%$ probability. Our attacks are tailored to three distance metrics used previously in the literature, and when compared to previous adversarial example generation algorithms, our attacks are often much more effective (and never worse). Furthermore, we propose using high-confidence adversarial examples in a simple transferability test we show can also be used to break defensive distillation. We hope our attacks will be used as a benchmark in future defense attempts to create neural networks that resist adversarial examples.
1. Introduction
Deep neural networks (DNNs) have revolutionized fields from computer vision to speech recognition. However, their susceptibility to adversarial examples—carefully crafted inputs designed to cause misclassification—remains a critical security flaw. Prior to this work, defensive distillation was considered a premier defense, purportedly reducing attack success rates to near zero. This paper investigates whether such defenses truly harden models or merely "mask" gradients to defeat existing attack tools. We introduce the "C&W attacks," a suite of optimization-based methods that systematically dismantle the perceived robustness of distilled networks.
2. Methodology: Adversarial Generation as Optimization
The authors reformulate the search for an adversarial example as a constrained optimization problem. The goal is to find a perturbation $\delta$ that minimizes a distance metric $D(x, x+\delta)$ while ensuring the model $C(x+\delta) = t$ for a target class $t$.
Technical Contributions
- Objective Function: The authors evaluated seven objective functions, finding that a loss based on "logits" (pre-softmax values) is far more effective than standard cross-entropy. This prevents the loss from "saturating," which often stalls other optimizers.
- Box Constraints: To keep pixel values within the valid $[0, 1]$ range, they utilized a
tanhchange of variables, allowing for the use of unconstrained optimizers like Adam. - Distance Metrics: Specific attacks were developed for $L_2$ (Euclidean distance), $L_0$ (number of pixels changed), and $L_\infty$ (maximum change to any pixel).
As shown in Figure 6, the optimization process relies on a constant $c$ to balance the trade-off between the distance metric and the success of the attack. By performing a binary search for $c$, the attack can find the minimal distortion required to achieve a 100% success rate.
3. Experimental Evaluation
The attacks were benchmarked against MNIST, CIFAR-10, and ImageNet (using Inception v3).
Key Results
- 100% Success Rate: The C&W attacks achieved a perfect success rate on distilled versions of MNIST and CIFAR-10, proving that defensive distillation does not provide significant security gains.
- Superior Efficiency: Compared to previous state-of-the-art attacks like JSMA or DeepFool, C&W found adversarial examples with significantly lower distortion.
- Targeted ImageNet Attacks: The authors demonstrated the first targeted $L_0$ attack on ImageNet, successfully changing the classification by modifying only a few pixels.
Figure 2 illustrates the visual quality of the generated examples. Despite the 100% success rate, the perturbations remain virtually invisible to the human eye, highlighting the gap between model robustness and human perception.
4. Discussion: The Illusion of Robustness
The primary insight of this work is that defensive distillation relies on gradient masking. It makes the model's gradients nearly zero at the input, which causes gradient-based attacks to fail not because the adversarial examples are gone, but because the "direction" toward them is hidden. By utilizing logits instead of the final softmax output, the C&W attack bypasses this masking.
Furthermore, the authors argue that defenses must be tested against transferability. A truly robust model should resist adversarial examples generated on a different, unprotected "substitute" model. The paper demonstrates that high-confidence adversarial examples can be used to break defenses even when the internal weights are unknown.
5. Conclusion
This paper serves as a cautionary tale for the machine learning community: a defense that breaks current attacks is not necessarily robust. The C&W attacks provide a new, more rigorous benchmark for evaluating neural network security. Future defenses must move beyond gradient masking and address the fundamental optimization landscape of neural networks to achieve true robustness.
Summary updated