TL;DR
This paper investigates black-box adversarial attacks, where attackers don't know the model's internals. They demonstrate that 2-sided stochastic gradient estimation methods are robust to hyperparameter choices and can efficiently generate adversarial examples with minimal model queries, challenging the notion that such attacks are inherently inefficient.
What Problem Does This Paper Address?
The field of adversarial machine learning has seen significant progress in developing attacks that can fool AI models. However, most successful attacks assume "white-box" access, meaning the attacker knows the model's architecture and weights. This allows them to directly compute gradients and craft highly effective adversarial examples. In contrast, "black-box" attacks, where the attacker only has query access to the model (e.g., getting predicted labels or probabilities), are generally considered more challenging and potentially less efficient. This paper addresses the efficiency and robustness of black-box attacks, specifically focusing on gradient estimation techniques when only model logits are available.
Methods
The core of the paper lies in exploring gradient estimation techniques for black-box attacks. When direct gradient computation isn't possible, attackers can approximate it by observing how model outputs change when inputs are slightly perturbed. The paper focuses on two main types of stochastic gradient estimation:
- 1-sided estimation: This method estimates the gradient by observing the change in model output for a single perturbation direction.
- 2-sided estimation: This method uses two perturbations (one positive and one negative) along a given direction to estimate the gradient.
The authors investigate how different choices of the perturbation magnitude ($\delta$) and the distribution of random perturbation directions ($\xi_i, \Delta_i$) affect the success rate and the number of queries required for an attack. They specifically highlight that the choice of the random direction distribution is largely unimportant, and that 2-sided methods are robust to the scaling parameter $\delta$.
Datasets
The experiments were conducted on the NIPS 2017 adversarial vision competition dataset, which consists of 1000 ImageNet-like images with dimensions 299x299x3. For each image, the least likely class was chosen as the adversarial target class.
Key Insights and Ideas
- Robustness of 2-sided estimation: The paper's most significant finding is that 2-sided stochastic gradient estimation methods are remarkably insensitive to the choice of the scaling parameter $\delta$. This means attackers can use these methods without needing to meticulously tune this hyperparameter, making them more practical.
- Efficiency of black-box attacks: Contrary to potential assumptions, the paper shows that black-box attacks using 2-sided gradient estimation can be highly effective and require a relatively small number of model queries.
- Unimportance of random direction distribution: The authors found that the specific distribution used to sample random perturbation directions (e.g., Gaussian, Bernoulli, Uniform) has little impact on the attack's success rate. This simplifies the attacker's strategy.
- Trade-offs with 1-sided estimation: While 1-sided estimation can potentially require fewer queries, its success is highly dependent on the careful selection of $\delta$, making it less robust in practice compared to 2-sided methods.
Key Results
The paper presents results in Table 1, comparing different estimation methods and $\delta$ values.
- 2-sided attacks (NES, RDSA, SPSA): All 2-sided methods achieved 100% success rates across various $\delta$ values (1e-2, 1e-3, 1e-4). The number of median queries varied but remained within a reasonable range, with RDSA (2-sided) at $\delta=1e-2$ requiring the fewest queries (13719).
- 1-sided SPSA: This method showed a stark contrast. At $\delta=1e-2$, success rates were very low (9.88%), requiring a large number of queries (50206). However, at $\delta=1e-4$, success rates approached perfect (99.89%) with a significantly reduced query count (10244), highlighting its sensitivity to $\delta$.
Limitations and Future Work
- Sensitivity of 1-sided methods: The paper explicitly notes that 1-sided estimation is heavily dependent on the choice of $\delta$. This suggests that while potentially efficient, it requires careful tuning and might not be as reliable as 2-sided methods in real-world scenarios.
- Reproducible code: The authors provide a link to reproducible code, which is a positive step for future research and validation.
- Future directions: While not explicitly detailed, the findings suggest that further research could explore optimizing query efficiency for black-box attacks, potentially by combining gradient estimation with other black-box strategies or by investigating adaptive query strategies. The robustness of 2-sided methods also opens avenues for developing more practical and less brittle adversarial attack tools.
Summary updated