Improving the Transferability of Adversarial Examples via Direction Tuning

Xiangyuan Yang, Jie Lin, Hanlin Zhang, Xinyu Yang, Peng Zhao

Abstract

In transfer-based adversarial attacks, adversarial examples are generated using surrogate models to fool unseen victim models. Current gradient-based methods face a fundamental trade-off: large update steps lead to significant direction deviation, while small steps cause update oscillation, both of which hinder convergence and transferability. This paper introduces the Direction Tuning Attack (DTA), which utilizes an inner sampling loop to stabilize gradient directions. Additionally, a network pruning (NP) method is proposed to smooth decision boundaries and further reduce oscillation. Experiments on ImageNet demonstrate that DTA improves the average attack success rate (ASR) from 87.9% to 94.5% on undefended models and from 69.1% to 76.2% on defense-hardened models compared to state-of-the-art attacks.

The Challenge of Adversarial Transferability

In the field of adversarial machine learning, a "transfer-based attack" is a powerful black-box technique where an attacker generates malicious inputs using a surrogate model they control, hoping these inputs will also trick a target victim model. The success of such attacks depends on the "transferability" of the adversarial examples.

The authors identify a "Goldilocks" problem in current gradient-based optimization strategies:

  1. Large steps allow for fast optimization but cause "convergence speed deviation," where the update direction drifts away from the steepest path toward the victim's failure point.
  2. Small steps might seem more precise, but they lead to "update oscillation." The optimization gets trapped in local noise or "zig-zags" in the loss landscape, preventing the attack from reaching a truly effective state.

Methodology

To resolve these issues, the researchers propose two primary innovations designed to align the surrogate model's gradients more closely with those of potential victim models.

Direction Tuning Attack (DTA)

DTA introduces an inner loop into the optimization process. Instead of a single large step based on one gradient calculation, the algorithm takes $K$ very small "sampling steps." It calculates the gradient at each of these small steps and averages them. This averaged gradient is then used for the actual update of the adversarial image. By averaging these probes, the attack cancels out the noise (oscillation) inherent in small steps while maintaining the directional alignment that large steps usually lose. A variant, VDTA, integrates this logic with variance tuning to further stabilize the optimization path.

Network Pruning (NP)

The authors observed that complex surrogate models often have "jagged" decision boundaries that contribute to gradient oscillation. They propose pruning redundant or unimportant neurons in the surrogate model during the attack process. This effectively "smooths" the decision boundary, making the resulting gradient directions more consistent and transferable across different architectures.

Experimental Results

The effectiveness of DTA and NP was evaluated on ImageNet using surrogate models like VGG16, ResNet50, and Inception-v3. The victim models included a wide array of architectures and eight advanced defense-hardened methods, such as Randomized Smoothing and JPEG compression.

Visual comparison of adversarial examples generated by DTA and VDTA with and without network pruning against baseline methods.

The results show a significant jump in ASR compared to previous state-of-the-art methods like MI-FGSM and VNI-FGSM. On undefended models, the average ASR improved to 94.5%. Against defense-hardened models, the ASR rose to 76.2%. Furthermore, DTA proved to be "plug-and-play," reaching up to 97.8% ASR when combined with other techniques like Input Diversity (DIM).

Conclusion

By focusing on gradient alignment and landscape smoothing, Direction Tuning Attack (DTA) significantly enhances the transferability of adversarial examples. While the inner loop increases computational cost (roughly 10x slower than standard iterative attacks), the substantial gains in success rates against both standard and robust models mark a significant advancement in black-box adversarial research. Future work may focus on achieving similar tuning effects with lower computational overhead.

Summary updated

Abstract

In the transfer-based adversarial attacks, adversarial examples are only generated by the surrogate models and achieve effective perturbation in the victim models. Although considerable efforts have been developed on improving the transferability of adversarial examples generated by transfer-based adversarial attacks, our investigation found that, the big deviation between the actual and steepest update directions of the current transfer-based adversarial attacks is caused by the large update step length, resulting in the generated adversarial examples can not converge well. However, directly reducing the update step length will lead to serious update oscillation so that the generated adversarial examples also can not achieve great transferability to the victim models. To address these issues, a novel transfer-based attack, namely direction tuning attack, is proposed to not only decrease the update deviation in the large step length, but also mitigate the update oscillation in the small sampling step length, thereby making the generated adversarial examples converge well to achieve great transferability on victim models. In addition, a network pruning method is proposed to smooth the decision boundary, thereby further decreasing the update oscillation and enhancing the transferability of the generated adversarial examples. The experiment results on ImageNet demonstrate that the average attack success rate (ASR) of the adversarial examples generated by our method can be improved from 87.9\% to 94.5\% on five victim models without defenses, and from 69.1\% to 76.2\% on eight advanced defense methods, in comparison with that of latest gradient-based attacks.
BibTeX
@article{yang2023improving,
  author = {Xiangyuan Yang and Jie Lin and Hanlin Zhang and Xinyu Yang and Peng Zhao},
  title = {{Improving the Transferability of Adversarial Examples via Direction Tuning}},
  journal = {Information Sciences},
  year = {2023},
  url = {https://openreview.net/forum?id=pEDYfK8xYb},
}
Figure