Abstract
Adversarial examples, generated by carefully crafted perturbation, have attracted considerable attention in research fields. Recent works have argued that the existence of the robust and non-robust features is a primary cause of the adversarial examples, and investigated their internal interactions in the feature space. In this paper, we propose a way of explicitly distilling feature representation into the robust and non-robust features, using Information Bottleneck. Specifically, we inject noise variation to each feature unit and evaluate the information flow in the feature representation to dichotomize feature units either robust or non-robust, based on the noise variation magnitude. Through comprehensive experiments, we demonstrate that the distilled features are highly correlated with adversarial prediction, and they have human-perceptible semantic information by themselves. Furthermore, we present an attack mechanism intensifying the gradient of non-robust features that is directly related to the model prediction, and validate its effectiveness of breaking model robustness.
TL;DR
Researchers have developed a way to explicitly separate "robust" and "non-robust" features in deep neural networks using the Information Bottleneck framework. Their findings reveal that non-robust features—the brittle patterns that lead to adversarial misclassification—actually contain human-perceptible semantic information and are the primary drivers behind why adversarial attacks work so effectively.
What Problem Does This Paper Address?
Deep Neural Networks (DNNs) are notoriously vulnerable to adversarial examples: images with tiny, invisible perturbations that cause a model to confidently misclassify them.
A leading theory suggests this happens because DNNs learn two types of features:
- Robust features: Patterns that align with human perception (e.g., the shape of a dog's ear).
- Non-Robust features: Brittle, statistical patterns that are highly predictive but easily manipulated by noise.
While this theory is widely accepted, it has been difficult to "see" these features separately within a model's intermediate layers. This paper addresses the gap by providing a mathematical method to distill these features, allowing researchers to study how they interact and how they contribute to a model's failure.
Methods
The authors utilize the Information Bottleneck (IB) principle to perform this distillation. The core idea of IB is to find a representation that keeps as much information about the target label ($Y$) as possible while throwing away unnecessary information from the input ($X$).
- Noise Intervention: They inject Gaussian noise into the feature units of a specific layer (usually the last convolutional layer).
- Sensitivity Analysis: They measure how much noise a feature unit can handle before it stops being useful for prediction.
- Robust features are those that remain predictive even under significant noise variation.
- Non-robust features are highly sensitive; even small amounts of noise "break" their predictive power.
- Dichotomization: By setting a threshold ($T$) based on the natural variation of the features, they categorize every feature unit as either robust or non-robust.
- NRF Attack: They introduce a new attack called the Non-Robust Feature Attack (NRF). Instead of just attacking the final output, this mechanism specifically targets and intensifies the gradients of the non-robust features to break model stability.
Datasets
The authors validated their approach using three standard image classification benchmarks: CIFAR-10, SVHN, and Tiny-ImageNet. They tested these against several model architectures, including VGG-16 and WideResNet (WRN-28-10).
Key Insights and Ideas
- Non-Robust Features are Semantic: Contrary to some previous beliefs that non-robust features are just "unintelligible noise," the authors' visualizations show that these features actually carry semantic meaning. For example, in an adversarial image of a "Bird" being misclassified as a "Deer," the distilled non-robust features actually look like a deer to the human eye.
- The Driver of Misclassification: The experiments show that adversarial predictions are almost entirely correlated with the non-robust features. When you isolate the robust features, the model remains remarkably accurate, even under heavy attack.
- The $\beta$ Dial: In the Information Bottleneck objective, the parameter $\beta$ acts as a control knob. Increasing $\beta$ forces the model to compress more information, effectively filtering out the non-robust "chatter" and focusing on the core robust features.
Key Results
- Feature Isolation: When the authors propagated only robust features, the model maintained nearly 100% accuracy on CIFAR-10, even when the input was an adversarial example. Conversely, propagating only non-robust features led to the exact misclassification intended by the attacker.
- Attack Effectiveness: The proposed NRF attack proved more potent than standard attacks like PGD or FGSM when targeting robustly trained models. On CIFAR-10, the NRF attack dropped a robust model's accuracy to 17.1%, whereas the standard PGD attack only dropped it to 51.6%.
- Visual Evidence: Using feature visualization techniques, the authors demonstrated that robust features highlight the "correct" object, while non-robust features highlight the "adversarial" object.
Limitations and Future Work
- Layer Specificity: The current work focuses primarily on the last convolutional layer. Future research should explore how these features evolve across all layers of a deep stack.
- Threshold Selection: The threshold $T$ used to separate features is determined empirically. A more theoretical or adaptive way to set this threshold could improve distillation.
- Defense Applications: The ability to isolate robust features is a clear stepping stone toward building more resilient "robust-only" classifiers and improving adversarial detection strategies.
Summary updated