TL;DR
This paper introduces REGroup, a novel defense mechanism against adversarial attacks on deep neural networks. REGroup leverages intermediate layer representations of a pre-trained network to build an ensemble of generative classifiers, which then aggregate their predictions using a Borda count-based ranking system, achieving state-of-the-art robustness without requiring any retraining.
What Problem Does This Paper Address?
Deep Neural Networks (DNNs) have achieved remarkable success in various computer vision tasks. However, their susceptibility to adversarial examples—subtly perturbed inputs designed to fool classifiers—poses a significant threat to their reliability, especially in safety-critical applications. Existing defense strategies often involve computationally intensive adversarial training or input transformations, which require retraining or fine-tuning the model. This paper addresses the need for a defense mechanism that is effective, computationally efficient, and model-agnostic, without the burden of retraining.
Methods
REGroup's core innovation lies in its use of intermediate layer representations and a novel aggregation strategy. The approach works as follows:
- Generative Classifiers from Intermediate Layers: For a pre-trained DNN, REGroup treats each of the top k layers as a generative classifier. These classifiers are modeled as class-conditional mixture distributions. Crucially, they are trained using only a small subset of clean training data and the pre-activation neural responses from that layer. The paper highlights that both positive and negative pre-activation values contain valuable information for classification.
- Rank-Ordering Preferences: Instead of directly predicting class probabilities, each layer-wise generative classifier outputs a ranking preference for all classes. This is achieved by computing the KL-divergence between the layer's response distribution and the test sample's response distribution.
- Borda Count Rank Aggregation: The ranking preferences from the k selected layers are then aggregated using the Borda count method. Borda count is a preferential voting system where each candidate (class) receives points based on how many other candidates it is ranked above. This ensemble approach leverages the collective "wisdom" of multiple layers to make a final, robust prediction.
The key novelty here is the shift from direct prediction to rank aggregation and the use of intermediate layer responses as the basis for generative classifiers, all without retraining the original network.
Datasets
- ImageNet: Used extensively for both training the generative classifiers (a subset of ~50,000 images) and for evaluating the defense's performance against various attacks. Specific subsets of the ImageNet validation set (V50K, V10K, V2K, V10C) are defined for evaluation.
- CIFAR-10: Used for smaller-scale experiments to demonstrate the approach's scalability and effectiveness on a different dataset.
No new benchmarks were introduced; the paper focuses on evaluating against existing adversarial attacks on standard datasets.
Key Insights and Ideas
- Intermediate Layers Hold Robustness: The paper's central discovery is that intermediate layers of a pre-trained DNN, when characterized statistically, inherently possess robustness to adversarial attacks. This robustness can be harnessed without modifying the original network.
- Positive and Negative Pre-activations Matter: Contrary to assumptions that only positive activations are informative, REGroup finds that negative pre-activation values also contain crucial semantic information that contributes to robust classification.
- Rank Aggregation for Consensus: Shifting from direct prediction to rank aggregation using Borda count provides a more robust decision-making process, similar to how human groups make collective decisions, making it less susceptible to individual layer failures.
- Attack-Agnostic Defense: REGroup's design, relying on statistical properties of intermediate layers and a general aggregation mechanism, makes it effective against a wide range of adversarial attacks, including both gradient-based and gradient-free methods.
Key Results
- State-of-the-Art Robustness: REGroup achieves state-of-the-art performance on the ImageNet validation set against various adversarial attacks, outperforming existing defenses like BaRT [41] and even showing competitive results against adversarial training methods.
- High Accuracy on Strong Attacks: For PGD attacks with a high perturbation strength ($\epsilon=16$), REGroup achieves a Top-1 accuracy of 22% on ResNet-50, significantly better than the 0% accuracy of the standard SoftMax classifier.
- Model-Agnostic: The defense works effectively with different network architectures, including ResNet-50 and VGG-19.
- No Retraining Required: A significant advantage is that REGroup does not require any retraining or fine-tuning of the pre-trained model, making it computationally efficient.
- Inference Time: While slightly slower than standard SoftMax (e.g., 0.16s vs 0.03s on GPU for ResNet-50), it is still practical for inference.
Limitations and Future Work
- Hyperparameter k: The choice of k (the number of top layers to aggregate) is a hyperparameter. The paper suggests a heuristic for selecting k (achieving at least 75% accuracy on clean samples) and shows that reasonable variations do not drastically impact performance. However, a substantial change in this criterion can degrade performance.
- Theoretical Analysis: The paper suggests that a deeper theoretical analysis of REGroup, potentially inspired by recent work on neurons as cooperating classifiers, would be a valuable future direction.
- Trade-off with Clean Accuracy: Like many robust defenses, REGroup shows a slight decrease in accuracy on clean, unperturbed images compared to the original SoftMax classifier, though it significantly improves robustness against adversarial examples. This trade-off is acknowledged and discussed in the context of existing literature.
Summary updated