Abstract
Although existing backdoor defenses have gained success in mitigating backdoor attacks, they still face substantial challenges. In particular, most of them rely on large amounts of clean data to weaken the backdoor mapping but generally struggle with residual trigger effects, resulting in persistently high attack success rates (ASR). Therefore, in this paper, we propose a novel Backdoor defense method based on Directional mapping module and adversarial Knowledge Distillation (BeDKD), which balances the trade-off between defense effectiveness and model performance using a small amount of clean and poisoned data. We first introduce a directional mapping module to identify poisoned data, which destroys clean mapping while keeping backdoor mapping on a small set of flipped clean data. Then, the adversarial knowledge distillation is designed to reinforce clean mapping and suppress backdoor mapping through a cycle iteration mechanism between trust and punish distillations using clean and identified poisoned data. Experimental results demonstrate that BeDKD surpasses the state-of-the-art defenses and reduces the ASR by 98% without significantly reducing the CACC.
Introduction
Deep Neural Networks (DNNs) are increasingly vulnerable to backdoor attacks, where malicious triggers (e.g., specific words or syntactic patterns) are injected into training data. An infected model performs normally on clean inputs but produces attacker-specified outputs when the trigger is present. Current defenses are categorized into data-level filtering and model-level erasure. However, these methods often suffer from a "defense trade-off": they either require prohibitive amounts of clean data or fail to fully eliminate the backdoor, leaving a high residual Attack Success Rate (ASR). BeDKD addresses these issues by achieving robust defense with minimal data and effectively "unlearning" the backdoor mapping.
Methodology
The BeDKD framework consists of two primary stages: Poisoned Data Identification and Adversarial Knowledge Distillation.
Directional Mapping Module (DMM)
To identify poisoned samples without extensive manual inspection, BeDKD utilizes a Directional Mapping Module. The DMM is initialized with the poisoned model's parameters and fine-tuned on a tiny subset of clean data with intentionally flipped labels. This process exploits the inherent stability of backdoor mappings; while the clean mapping is weakened by the noisy labels, the trigger-to-target relationship remains robust. By calculating the Mean Error of Probability Distributions (MEPD) between the original model and the DMM, the system can accurately isolate poisoned samples from the training set.
Adversarial Knowledge Distillation (AKD)
Following identification, the defense employs a cycle iteration mechanism between two distillation processes:
- Trust Distillation: A student model learns the correct clean mapping from a teacher model using a small clean dataset.
- Punish Distillation: The student model is explicitly penalized for following the backdoor mapping on the identified poisoned samples through a specialized penalty loss function. This dual-path approach ensures the model retains primary task performance while aggressively suppressing the backdoor.
Experimental Results
BeDKD was evaluated against six diverse attacks (word-level, syntactic, and style-based) across the SST2, AGnews, and OLID datasets.
The results indicate that BeDKD consistently reduces ASR to near-zero levels (averaging 0.81% to 1.79% across datasets) while maintaining Clean Accuracy (CACC) within 1% of the original performance. Notably, BeDKD demonstrates superior efficiency; for instance, on SST2, it completes the defense in 232 seconds, significantly faster than W2SDefense (4,000+ seconds) or TextGuard (30,000+ seconds). Furthermore, the method proves robust even when the clean data used for distillation is from a different domain than the training data.
Conclusion
BeDKD represents a significant advancement in model-level backdoor defense by leveraging the stability of backdoor mappings for identification and adversarial distillation for erasure. It breaks the traditional data-dependency barrier, requiring only a small fraction of data to achieve state-of-the-art results. Future work will focus on extending these techniques to generative Large Language Models (LLMs).
Summary updated