Lipschitz-certifiable training with a tight outer bound

Sungyoon Lee, Jaewook Lee, Saerom Park

Abstract

Verifiable training is a promising research direction for training a robust network. However, most verifiable training methods are slow or lack scalability. In this study, we propose a fast and scalable certifiable training algorithm based on Lipschitz analysis and interval arithmetic. Our certifiable training algorithm provides a tight propagated outer bound by introducing the box constraint propagation (BCP), and it efficiently computes the worst logit over the outer bound. In the experiments, we show that BCP achieves a tighter outer bound than the global Lipschitz-based outer bound. Moreover, our certifiable training algorithm is over 12 times faster than the state-of-the-art dual relaxation-based method; however, it achieves comparable or better verification performance, improving natural accuracy. Our fast certifiable training algorithm with the tight outer bound can scale to Tiny ImageNet with verification accuracy of 20.1\%(l2-perturbation of 36/255). Our code is available at \url {https://github. com/sungyoon-lee/bcp}.
BibTeX
@inproceedings{lee2020lipschitz,
  author = {Sungyoon Lee and Jaewook Lee and Saerom Park},
  title = {{Lipschitz-certifiable training with a tight outer bound}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2020},
  url = {https://openreview.net/forum?id=G8jarY0zeq},
}

Loading related content...

Figure