On Connections between Regularizations for Improving DNN Robustness

Yiwen Guo, Long Chen, Yurong Chen, Changshui Zhang

Abstract

This paper analyzes regularization terms proposed recently for improving the adversarial robustness of deep neural networks (DNNs), from a theoretical point of view. Specifically, we study possible connections between several effective methods, including input-gradient regularization, Jacobian regularization, curvature regularization, and a cross-Lipschitz functional. We investigate them on DNNs with general rectified linear activations, which constitute one of the most prevalent families of models for image classification and a host of other machine learning applications. We shed light on essential ingredients of these regularizations and re-interpret their functionality. Through the lens of our study, more principled and efficient regularizations can possibly be invented in the near future.
BibTeX
@article{guo2021connections,
  author = {Yiwen Guo and Long Chen and Yurong Chen and Changshui Zhang},
  title = {{On Connections between Regularizations for Improving DNN Robustness}},
  journal = {IEEE Trans. Pattern Anal. Mach. Intell.},
  year = {2021},
  url = {https://openreview.net/forum?id=-Bwedr0uP8l},
}
Figure