Introduction
To evaluate and enhance the robustness of deep learning models, researchers utilize adversarial examples—images containing subtle, human-imperceptible perturbations designed to deceive classifiers. A primary challenge in this field is the "theory-practice gap" found in state-of-the-art attacks like the Momentum Iterative Fast Gradient Sign Method (MI-FGSM). While theoretical convergence for momentum-based optimization typically requires a monotonic parameter schedule, practical implementations often rely on constant values, which lack a rigorous theoretical foundation. Furthermore, recent optimization research indicates that sign-based methods, common in adversarial attacks, frequently fail to converge to true optima even in simple convex settings, limiting their effectiveness in black-box transfer scenarios.
Methodology: The AdaMSI-FGM Framework
This paper introduces the Adaptive Momentum and Step-size Iterate Fast Gradient Method (AdaMSI-FGM), a novel approach that bridges the gap between theoretical optimization and practical performance. The method is built upon two core innovations:
- Adaptive Momentum Parameter ($\beta_{1,t}$): Instead of adhering to a restrictive monotonic schedule or a fixed constant, the authors propose a parameter that adapts dynamically to the real-time variation of the gradient's $L_1$ norm. This flexibility allows for a convergence guarantee without sacrificing practical performance.
- Adaptive Step-Size via Second Moment Estimate (SME): Recognizing that the standard
sign(gradient)operation is essentially a rigid form of adaptive step-size, the authors replace it with a Second Moment Estimate (SME) similar to the Adam optimizer. This transition avoids the non-convergence issues inherent in sign-based methods.
Mathematically, the authors demonstrate that AdaMSI-FGM achieves a data-dependent regret bound of $O(\sqrt{T})$. Notably, this bound is tighter than traditional Adam-type algorithms as it eliminates the logarithmic factor that typically slows theoretical convergence.
Convergence Analysis
A key insight of this work is the treatment of the sign function as a crude adaptive step-size. By employing more sophisticated optimization tools like SME, the attack achieves superior convergence properties.
As illustrated in the convergence plots, traditional sign-based methods such as the Basic Iterative Method (BIM) tend to plateau quickly, failing to further minimize the loss function. In contrast, AdaMSI-FGM continues to decline, reaching a significantly lower and more optimal value within the same number of iterations. This reliable convergence on the surrogate (source) model is a primary driver for the method's improved transferability to unknown target models.
Experimental Results
The method was rigorously evaluated using 500 randomly selected images from the ImageNet (ILSVRC 2012) validation set against eight pre-trained models, including standard architectures (e.g., ResNet34, EfficientNet-b0) and adversarially trained robust models (e.g., Inc-v3ens3).
- Success Rates: In black-box transfer scenarios, AdaMSI-FGM consistently outperformed classic attacks like PGD and AutoAttack by margins of 2% to 17.8% in Attack Success Rate (ASR).
- Robustness Benchmarking: When compared to the industry-standard MI-FGSM, AdaMSI-FGM showed superior performance, particularly against models specifically hardened to resist adversarial noise.
- Imperceptibility: Despite the increased attack potency, the generated perturbations remain invisible to the human eye, preserving the stealth required for effective adversarial testing.
Conclusion and Future Work
AdaMSI-FGM demonstrates that adaptive momentum and step-size schemes can significantly enhance the effectiveness of adversarial attacks while providing solid theoretical convergence. While this study focused on the $L_\infty$ norm and non-targeted attacks, future research could extend these adaptive principles to $L_2$ metrics and targeted attack scenarios. Additionally, the inherent flexibility of the method suggests it could be synergistically combined with other techniques like Input Diversity (DI) or Translation Invariance (TI) to further push the boundaries of adversarial robustness testing.
Summary updated