DERD: Data-free Adversarial Robustness Distillation through Self-adversarial Teacher Group

Yuhang Zhou, Yushu Zhang, Leo Yu Zhang, Zhongyun Hua

Abstract

Computer vision models based on deep neural networks are proven to be vulnerable to adversarial attacks. Robustness distillation, as a countermeasure, takes both robustness challenges and efficiency challenges of edge models into consideration. However, most existing robustness distillations are data-driven, which can hardly be deployed in the data-privacy scenarios. Also, the trade-off between robustness and accuracy tends to transfer from the teacher to the student along with the robustness, and there has been no discussion on mitigating this trade-off in the data-free scenario yet. In this paper, we propose a Data-free Experts-guided Robustness Distillation (DERD) to extend robustness distillation to data-free paradigm, which offers three advantages: (1) Dual-level adversarial learning strategy achieves robustness distillation without the real data. (2) Expert-guided distillation strategy brings a better trade-off to the student model. (3) A novel stochastic gradient aggregation module reconciles the task conflicts of the multi-teacher from a consistency perspective. Extensive experiments demonstrate that the proposed DERD can even achieve comparable results to data-driven methods.
BibTeX
@inproceedings{zhou2024derd,
  author = {Yuhang Zhou and Yushu Zhang and Leo Yu Zhang and Zhongyun Hua},
  title = {{DERD: Data-free Adversarial Robustness Distillation through Self-adversarial Teacher Group}},
  booktitle = {ACM Multimedia},
  year = {2024},
  doi = {10.1145/3664647.3680796},
}

Loading related content...

Figure