UMA: Facilitating Backdoor Scanning via Unlearning-Based Model Ablation

Yue Zhao, Congyi Li, Kai ChenAAAI 2024 · OpenReview

Introduction

Deep Neural Networks (DNNs) are increasingly vulnerable to backdoor attacks, where hidden triggers cause misclassification while maintaining normal behavior on clean inputs. Detecting these "trojans" is challenging because advanced attacks often entangle backdoor features with the model's inherent features or use complex, non-patch-based triggers. Current scanners frequently struggle to differentiate between a true backdoor and a Universal Adversarial Perturbation (UAP)—a natural glitch inherent to the model's task.

Methodology: Unlearning-based Model Ablation (UMA)

UMA introduces a novel approach to backdoor scanning by leveraging machine unlearning to isolate hidden triggers. The pipeline consists of three primary stages:

  1. Model Ablation via Unlearning: The core innovation involves forcing the model to "forget" a target class. By minimizing the model's ability to recognize natural features of that class, UMA effectively removes the features that lead to false positives (UAPs). To prevent accidental erasure of the backdoor trigger, which typically relies on low-level visual primitives, the early layers of the model are frozen during this process.
  2. Dynamic Trigger Optimization: Following ablation, UMA reverse-engineers potential triggers using a dynamic hyperparameter ($\lambda$). This parameter self-adjusts to find the "tightest" possible trigger, ensuring high attack success rates while maintaining size constraints without manual tuning.
  3. Transferability-based Filter: To eliminate remaining false positives, UMA evaluates the reversed trigger on a clean version of the model. Since UAPs exploit general task features, they tend to transfer across models. In contrast, true backdoor triggers are specific to the poisoned model and fail to transfer, allowing for clear distinction.
Overview of the UMA pipeline and its ability to distinguish backdoor triggers from natural features compared to previous methods.

Experimental Evaluation

The researchers evaluated UMA against a massive suite of 1,700 models, including those from the 2022 Trojan Detection Challenge (TDC) and an advanced attack suite featuring composite, reflection, and frequency-domain attacks. The study spanned six architectures, including standard CNNs, WideResNets, and Vision Transformers (ViT).

Key Results

UMA significantly outperformed state-of-the-art baselines such as Neural Cleanse (NC), ABS, K-Arm, and MNTD. On the TDC datasets, UMA achieved an average AUC-ROC of 91% and a detection accuracy of 86.6%, whereas the next best baseline (Neural Cleanse) reached only 71.8% AUC-ROC.

Performance comparison of UMA against state-of-the-art backdoor detection methods across multiple datasets.

Beyond accuracy, UMA demonstrated remarkable efficiency. For a CIFAR-10 model, UMA completed detection in 195 seconds, significantly faster than Neural Cleanse (800s) and ABS (333s).

Discussion and Insights

The success of UMA stems from treating function ablation as a proxy for feature ablation. By removing the model's knowledge of a specific task (the class), UAPs are naturally suppressed, leaving the "unnatural" backdoor trigger exposed. This insight highlights that UAPs are essentially inherent features of the model's task.

However, UMA does have limitations. It requires a small set of clean data for the unlearning process and the transferability filter. Additionally, its sensitivity may vary based on the number of classes in the model, as unlearning a single class represents a larger portion of the model's knowledge in low-class-count scenarios. Future work may explore methods to reduce clean data dependency or further refine the ablation process for perfectly entangled triggers.

Summary updated

Abstract

Recent advances in backdoor attacks, like leveraging complex triggers or stealthy implanting techniques, have introduced new challenges in backdoor scanning, limiting the usability of Deep Neural Networks (DNNs) in various scenarios. In this paper, we propose Unlearning-based Model Ablation (UMA), a novel approach to facilitate backdoor scanning and defend against advanced backdoor attacks. UMA filters out backdoor-irrelevant features by ablating the inherent features of the target class within the model and subsequently reveals the backdoor through dynamic trigger optimization. We evaluate our method on 1700 models (700 benign and 1000 trojaned) with 6 model structures, 7 different backdoor attacks and 4 datasets. Our results demonstrate that the proposed methodology effectively detect these advanced backdoors. Specifically, our method can achieve 91% AUC-ROC and 86.6% detection accuracy on average, which outperforms the baselines, including Neural Cleanse, ABS, K-Arm and MNTD.
BibTeX
@inproceedings{zhao2024uma,
  author = {Yue Zhao and Congyi Li and Kai Chen},
  title = {{UMA: Facilitating Backdoor Scanning via Unlearning-Based Model Ablation}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2024},
  url = {https://openreview.net/forum?id=w9ia1Kxg4o},
}
Figure