TL;DR
This paper proposes a novel, training-free defense against query-based adversarial attacks on deep neural networks. By dynamically activating existing defenses only for low-confidence predictions, the authors achieve a significantly better accuracy-robustness tradeoff, outperforming previous methods without requiring model retraining.
What Problem Does This Paper Address?
Deep neural networks (DNNs) are increasingly used in critical applications, but they are vulnerable to adversarial attacks. Adversarial samples are inputs subtly modified to fool the model, leading to misclassifications. While defenses exist, they often suffer from a significant drop in accuracy on clean, unperturbed data. This is particularly problematic for query-based black-box attacks, where attackers have limited knowledge of the model and can only query its outputs. These attacks exploit low-confidence regions of the model's decision boundary. Existing defenses, like random noise, improve robustness but at the cost of clean accuracy, creating a difficult tradeoff.
Methods
The core idea of this paper is to leverage the observation that genuine inputs are typically classified with high confidence, while adversarial attacks must explore low-confidence regions. The proposed method, parameterized by a confidence threshold $\tau$, dynamically activates an existing inference-time defense (e.g., random noise, random cropping, JPEG compression) only when the model's confidence in its prediction falls below this threshold. For high-confidence inputs, the original, undefended model is used. This approach is entirely training-free and can be applied to any existing inference-time defense. The authors also discuss theoretical motivations based on model calibration and the nature of binary search in decision-based attacks.
Datasets
The authors evaluate their approach on the following standard datasets:
- CIFAR-10
- CIFAR-100
- ImageNet
Key Insights and Ideas
- Low-confidence regions are the key: Adversarial attacks, especially decision-based ones, rely on exploring low-confidence regions to find the decision boundary. Genuine inputs, however, are usually classified with high confidence.
- Dynamic defense activation is sufficient: Activating defenses only in these low-confidence regions is enough to disrupt the attack without significantly impacting clean accuracy.
- Training-free is a major advantage: The ability to apply this defense strategy to pre-trained models without retraining is a significant practical benefit.
- Confidence threshold is crucial: The confidence threshold $\tau$ acts as a knob to tune the accuracy-robustness tradeoff, allowing for a more nuanced approach than simply always applying a defense.
Key Results
The paper demonstrates significant improvements in accuracy-robustness tradeoffs across various datasets and defenses. For instance:
- On CIFAR-10, their approach improved robust accuracy by up to 9% against the PSJA attack with a negligible impact on clean accuracy (at most 2%).
- Against the SurFree attack, they reported robustness improvements of up to 34% with almost no degradation of main task accuracy.
- When compared to baseline defenses (where the defense is always active, equivalent to $\tau=1.0$), their method achieved better Pareto frontiers, meaning higher robust accuracy for a given clean accuracy, or vice-versa. For example, on CIFAR-10 with the RND defense against PSJA, they improved CA by 29% while decreasing RA by only 1% compared to the baseline.
Limitations and Future Work
- Model Calibration: While the approach is largely agnostic to model calibration, the optimal threshold $\tau$ might vary. The paper includes a calibration step (temperature scaling) to ensure consistent performance.
- ImageNet Performance: For ImageNet, the authors note that RA can decrease with increasing parameters $\nu$ and $\tau$. They attribute this to the larger input space and higher number of classes, where introduced noise might inadvertently facilitate label flips.
- Future Work: The authors hope their work will motivate further research into similar dynamic defense activation strategies. They also suggest exploring other lightweight inference-time defenses.
Summary updated