TL;DR
This paper tackles the challenge of "hard-label" adversarial attacks, where attackers only know the top predicted label of a target model. They introduce a novel approach that leverages "transfer-based priors" from other models to significantly improve the efficiency of these attacks, requiring fewer queries to fool the target model. This is crucial for developing more robust AI systems by understanding their vulnerabilities.
What Problem Does This Paper Address?
Adversarial attacks are a major concern for AI security, aiming to trick machine learning models into making incorrect predictions with subtle, imperceptible changes to input data. "Hard-label" attacks are particularly challenging and practical because they only reveal the top predicted label, not the model's confidence scores. This limited information makes it difficult to craft effective attacks. Existing methods often struggle with either high query costs (requiring many interactions with the target model) or low accuracy. This paper focuses on improving the query efficiency of hard-label attacks by using knowledge from other, more accessible models (surrogate models) to guide the attack process.
Methods
The core of this paper lies in enhancing the "ray search" procedure for hard-label attacks. This procedure aims to find the shortest path (ray) from a benign image to the adversarial region (where the model misclassifies).
- Ray Search Optimization: The problem is reformulated as minimizing the distance along a ray direction. This is typically done using zeroth-order optimization, which estimates gradients without direct access to model internals.
- Transfer-based Priors: The key innovation is incorporating "transfer-based priors." These are essentially estimated gradients from surrogate models that are trained on similar data. The authors propose a method to derive these priors by using a surrogate loss function that approximates the gradient of the objective function.
- Prior-Guided Gradient Estimation: The paper introduces two novel algorithms: Prior-Sign-OPT and Prior-OPT. These algorithms intelligently combine the transfer-based priors with random vectors. They use a subspace projection approach to approximate the true gradient, giving more weight to the priors when they are deemed reliable. Prior-OPT, in particular, adaptively weighs priors and uses a more sophisticated estimation for them compared to random vectors.
- Theoretical Analysis: The authors provide theoretical guarantees for their proposed methods, analyzing the expected cosine similarity between their estimated gradients and the true gradient. This helps to understand why their approach is more effective.
Datasets
The experiments were conducted on two widely used image datasets:
- ImageNet: A large-scale dataset with diverse images.
- CIFAR-10: A smaller dataset with 10 classes of images.
Key Insights and Ideas
- Leveraging Surrogate Knowledge is Key: The paper demonstrates that knowledge from surrogate models can significantly boost the efficiency of hard-label attacks, even when the surrogate models are not perfectly aligned with the target model.
- Prior Quality Matters: The effectiveness of the priors is directly related to their accuracy (measured by cosine similarity). The proposed methods can adapt to varying prior quality.
- Prior-OPT's Adaptive Approach: Prior-OPT's ability to treat priors and random vectors differently, and to adaptively weigh priors, is crucial for achieving superior performance.
- Theoretical Foundation for Practical Gains: The theoretical analysis provides a strong foundation for the empirical improvements observed, explaining why the proposed methods work better.
Key Results
- Significant Query Efficiency Improvement: The proposed Prior-OPT and Prior-Sign-OPT methods significantly outperform 11 state-of-the-art hard-label attacks on ImageNet and CIFAR-10 datasets in terms of query efficiency. For instance, they achieve much lower mean l2 distortions with fewer queries.
- Superior Performance with Multiple Priors: Using multiple surrogate models (priors) consistently leads to better attack performance.
- PGD Initialization Benefits: Initializing the attack with Projected Gradient Descent (PGD) on a surrogate model further enhances performance, especially in early iterations.
- Targeted vs. Untargeted Attacks: Prior-OPT generally performs better in untargeted attacks, while Prior-Sign-OPT shows advantages in targeted attacks with more queries due to its query-efficient sign-based estimation.
Limitations and Future Work
- Computational Overhead: While query efficiency is improved, the introduction of binary search for priors in Prior-OPT does add some computational overhead compared to simpler methods like Sign-OPT. However, this is considered a worthwhile trade-off for improved attack performance.
- Prior Quality Assumption: The theoretical analysis relies on the assumption that priors have some level of informativeness (non-zero cosine similarity with the true gradient). If priors are completely uninformative, the performance might degrade.
- Future Work: The authors suggest exploring further improvements on the CIFAR-10 dataset and investigating more advanced methods for incorporating priors.
Summary updated