Abstract: Adversarial attacks can misguide deep neural networks (DNNs) by adding small-magnitude perturbations to normal examples, primarily determined by the gradient of the loss function with respect to inputs. Previously, various strategies have been proposed to enhance the performance of adversarial attacks. However, all these methods only utilize gradients in the present and past. Until now, the trend of gradient change in the future (i.e., the derivative of the gradient) has not been considered. Inspired by the classic proportional-integral-derivative (PID) controller, we propose a new PID-based approach for generating adversarial examples. The gradients in the present and past, and the derivative of the gradient are considered, corresponding to the P, I, and D components, respectively. Extensive experiments consistently demonstrate that our method achieves higher attack success rates and exhibits better transferability compared with state-of-the-art gradient-based adversarial attacks.
Introduction
Adversarial attacks work by making tiny, invisible changes to an image that cause a Deep Neural Network (DNN) to misclassify it. Most state-of-the-art attacks rely on the gradient of the loss function to find these perturbations. However, existing methods like the Fast Gradient Sign Method (FGSM) and its iterative versions (MI-FGSM, NI-FGSM) have a "memory" problem. They look at the current gradient (the "present") and accumulated past gradients (the "past"), but they ignore the trend of gradient change (the "future"). This often leads to adversarial examples that "overfit" to the specific architecture of the source model, making them less effective when tested against different "black-box" models or models with active defenses.
The PID Framework for Adversarial Generation
The authors propose a PID-based approach to optimization, mapping the components of a classic PID controller to the adversarial generation process:
- Proportional (P): The current gradient (used by basic FGSM).
- Integral (I): The accumulation of past gradients (used by momentum-based attacks like MI-FGSM).
- Derivative (D): The new addition—the difference between gradients, which represents the "velocity" or trend of the loss surface.
The core innovation lies in the MID-FGSM and NID-FGSM algorithms. To calculate the "D" term without knowing the actual future, the authors use an intermediate "backward-tuned" state. By comparing the gradient at the current point with the gradient at this slightly adjusted point, they estimate the derivative. This framework treats the iterative generation of an adversarial example as a feedback control system, preventing the perturbation from falling into poor local maxima that do not generalize to other models.
Efficiency and Performance
Surprisingly, adding the complexity of the derivative term actually makes the attack more efficient. The predictive power of the derivative acts as a look-ahead mechanism, finding smoother paths on the loss landscape. The authors found that a PID-based attack with only 5 iterations often outperformed standard momentum attacks that used 10 iterations.
Key Results and Transferability
The PID-based approach consistently outperforms previous state-of-the-art methods. In black-box attacks—where an attack is generated on one model and tested on another—the PID-based approach improved success rates by 1% to 6%. Furthermore, the framework is highly extensible and can be combined with other common techniques like Diverse Inputs (DI), Translation-Invariance (TI), and Scale-Invariance (SI).
When combined with these data augmentation tricks (SI-TI-DI), the PID-based attacks achieved massive gains against advanced defense models. The adversarial examples generated via PID control were much more robust, maintaining high failure rates even when transferred to models the attacker had never seen, proving they capture a more general weakness in neural network logic.
Conclusion
By incorporating the trend of gradient change, the PID-based approach provides a more robust and transferable method for generating adversarial examples. This method not only improves success rates against both normally trained and defense-hardened models but also does so with greater computational efficiency. Future work may explore applying PID control to the standard training of DNNs to improve their inherent robustness.
Summary updated