DERD: Data-free Adversarial Robustness Distillation through Self-adversarial Teacher Group
Lists
Only you see this
Abstract
Computer vision models based on deep neural networks are proven to be vulnerable to adversarial attacks. Robustness distillation, as a countermeasure, takes both robustness challenges and efficiency challenges of edge models into consideration. However, most existing robustness distillations are data-driven, which can hardly be deployed in the data-privacy scenarios. Also, the trade-off between robustness and accuracy tends to transfer from the teacher to the student along with the robustness, and there has been no discussion on mitigating this trade-off in the data-free scenario yet. In this paper, we propose a Data-free Experts-guided Robustness Distillation (DERD) to extend robustness distillation to data-free paradigm, which offers three advantages: (1) Dual-level adversarial learning strategy achieves robustness distillation without the real data. (2) Expert-guided distillation strategy brings a better trade-off to the student model. (3) A novel stochastic gradient aggregation module reconciles the task conflicts of the multi-teacher from a consistency perspective. Extensive experiments demonstrate that the proposed DERD can even achieve comparable results to data-driven methods.
BibTeX
@inproceedings{zhou2024derd,
author = {Yuhang Zhou and Yushu Zhang and Leo Yu Zhang and Zhongyun Hua},
title = {{DERD: Data-free Adversarial Robustness Distillation through Self-adversarial Teacher Group}},
booktitle = {ACM Multimedia},
year = {2024},
doi = {10.1145/3664647.3680796},
}Loading related content...