PID-Based Approach to Adversarial Attacks

Chen Wan, Biaohua Ye, Fangjun Huang

Abstract: Adversarial attacks can misguide deep neural networks (DNNs) by adding small-magnitude perturbations to normal examples, primarily determined by the gradient of the loss function with respect to inputs. Previously, various strategies have been proposed to enhance the performance of adversarial attacks. However, all these methods only utilize gradients in the present and past. Until now, the trend of gradient change in the future (i.e., the derivative of the gradient) has not been considered. Inspired by the classic proportional-integral-derivative (PID) controller, we propose a new PID-based approach for generating adversarial examples. The gradients in the present and past, and the derivative of the gradient are considered, corresponding to the P, I, and D components, respectively. Extensive experiments consistently demonstrate that our method achieves higher attack success rates and exhibits better transferability compared with state-of-the-art gradient-based adversarial attacks.

Introduction

Adversarial attacks work by making tiny, invisible changes to an image that cause a Deep Neural Network (DNN) to misclassify it. Most state-of-the-art attacks rely on the gradient of the loss function to find these perturbations. However, existing methods like the Fast Gradient Sign Method (FGSM) and its iterative versions (MI-FGSM, NI-FGSM) have a "memory" problem. They look at the current gradient (the "present") and accumulated past gradients (the "past"), but they ignore the trend of gradient change (the "future"). This often leads to adversarial examples that "overfit" to the specific architecture of the source model, making them less effective when tested against different "black-box" models or models with active defenses.

The PID Framework for Adversarial Generation

The authors propose a PID-based approach to optimization, mapping the components of a classic PID controller to the adversarial generation process:

  • Proportional (P): The current gradient (used by basic FGSM).
  • Integral (I): The accumulation of past gradients (used by momentum-based attacks like MI-FGSM).
  • Derivative (D): The new addition—the difference between gradients, which represents the "velocity" or trend of the loss surface.

The core innovation lies in the MID-FGSM and NID-FGSM algorithms. To calculate the "D" term without knowing the actual future, the authors use an intermediate "backward-tuned" state. By comparing the gradient at the current point with the gradient at this slightly adjusted point, they estimate the derivative. This framework treats the iterative generation of an adversarial example as a feedback control system, preventing the perturbation from falling into poor local maxima that do not generalize to other models.

Efficiency and Performance

Surprisingly, adding the complexity of the derivative term actually makes the attack more efficient. The predictive power of the derivative acts as a look-ahead mechanism, finding smoother paths on the loss landscape. The authors found that a PID-based attack with only 5 iterations often outperformed standard momentum attacks that used 10 iterations.

Success rate vs. iterations

Key Results and Transferability

The PID-based approach consistently outperforms previous state-of-the-art methods. In black-box attacks—where an attack is generated on one model and tested on another—the PID-based approach improved success rates by 1% to 6%. Furthermore, the framework is highly extensible and can be combined with other common techniques like Diverse Inputs (DI), Translation-Invariance (TI), and Scale-Invariance (SI).

When combined with these data augmentation tricks (SI-TI-DI), the PID-based attacks achieved massive gains against advanced defense models. The adversarial examples generated via PID control were much more robust, maintaining high failure rates even when transferred to models the attacker had never seen, proving they capture a more general weakness in neural network logic.

Success rates for combined attacks

Conclusion

By incorporating the trend of gradient change, the PID-based approach provides a more robust and transferable method for generating adversarial examples. This method not only improves success rates against both normally trained and defense-hardened models but also does so with greater computational efficiency. Future work may explore applying PID control to the standard training of DNNs to improve their inherent robustness.

Summary updated

Abstract

Adversarial attack can misguide the deep neural networks (DNNs) with adding small-magnitude perturbations to normal examples, which is mainly determined by the gradient of the loss function with respect to inputs. Previously, various strategies have been proposed to enhance the performance of adversarial attacks. However, all these methods only utilize the gradients in the present and past to generate adversarial examples. Until now, the trend of gradient change in the future (i.e., the derivative of gradient) has not been considered yet. Inspired by the classic proportional-integral-derivative (PID) controller in the field of automatic control, we propose a new PID-based approach for generating adversarial examples. The gradients in the present and past, and the derivative of gradient are considered in our method, which correspond to the components of P, I and D in the PID controller, respectively. Extensive experiments consistently demonstrate that our method can achieve higher attack success rates and exhibit better transferability compared with the state-of-the-art gradient-based adversarial attacks. Furthermore, our method possesses good extensibility and can be applied to almost all available gradient-based adversarial attacks.
BibTeX
@inproceedings{wan2021pid,
  author = {Chen Wan and Biaohua Ye and Fangjun Huang},
  title = {{PID-Based Approach to Adversarial Attacks}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  url = {https://openreview.net/forum?id=A4PHyuzJtI},
}
Figure