Backdoor Attacks via Machine Unlearning

Zihao Liu, Tianhao Wang, Mengdi Huai, Chenglin Miao

TL;DR

This paper explores a novel security threat: using machine unlearning, a privacy-preserving technique, to inject backdoor attacks into trained models. The authors propose two methods for attackers to achieve this, either by strategically unlearning their own data or by poisoning a small subset of data and then using unlearning to activate the backdoor, demonstrating significant attack success with minimal impact on benign model performance.

What Problem Does This Paper Address?

Machine unlearning is a rapidly developing field focused on efficiently removing specific data points from a trained machine learning model, often to comply with privacy regulations like GDPR. While existing research primarily focuses on the effectiveness and efficiency of unlearning, this paper highlights a critical security vulnerability: the potential for malicious actors to exploit machine unlearning processes to introduce backdoor attacks. A backdoor attack subtly alters a model's behavior, causing it to misclassify inputs that contain a specific "trigger," while otherwise performing normally. The paper investigates whether attackers can leverage unlearning requests to inject these backdoors, a scenario not previously explored, and how this could be achieved stealthily.

Methods

The paper proposes two primary attack strategies:

  • Attack without Poisoning (AwoP): This is the more challenging scenario where the attacker does not poison the original training data. Instead, the attacker strategically selects a subset of their own training data to be unlearned. By carefully choosing which data to "forget," the attacker aims to manipulate the unlearning process to embed a backdoor. This involves solving an optimization problem to balance maximizing attack utility (successful backdoor activation) with minimizing the unlearning set size for stealth.
  • Attack with Poisoning (AwP): This approach involves two stages. First, the attacker poisons a small number of training instances with a pre-defined trigger. Crucially, this poisoning is designed to be dormant and not immediately affect the model's performance. In the second stage, the attacker uses machine unlearning, similar to the AwoP strategy, to amplify the effect of the injected trigger, thereby activating the backdoor in the unlearned model. This method offers more flexibility in choosing the backdoor trigger.

Both attacks are formulated as optimization problems. To handle the discrete nature of selecting data for unlearning, the authors use a continuous, differentiable sigmoid function to approximate the selection probability, allowing for gradient-based optimization. For the black-box setting, where the attacker lacks model architecture knowledge, they propose training a "shadow model" using knowledge distillation to mimic the target model's behavior and derive attack parameters.

Datasets

The paper evaluates its proposed attacks using two standard image classification datasets:

  • CIFAR-10: A widely used dataset with 10 classes and 50,000 training images.
  • TinyImageNet: A larger dataset with 200 classes and 100,000 training images. For experiments, 100 classes of TinyImageNet were randomly selected.

The authors do not introduce new datasets but rather use these established benchmarks to evaluate their attack methodologies against various machine unlearning algorithms and model architectures.

Key Insights and Ideas

  • Unlearning as an Attack Vector: The most significant insight is that machine unlearning, intended for privacy, can be repurposed as a potent tool for sophisticated backdoor attacks. This flips the narrative from unlearning as a defense to unlearning as an offensive mechanism.
  • Stealthy Backdoor Injection: The proposed methods achieve high attack success rates while maintaining good benign accuracy, indicating that the backdoor is effectively hidden and the model's normal functionality is largely preserved. This stealth is crucial for evading detection.
  • Minimal Data Unlearning Required: The attacks are effective even when unlearning a very small subset of data, making them efficient and harder to detect by monitoring the volume of unlearning requests.
  • Flexibility in Attack Design: The two proposed methods cater to different attacker capabilities, from those with no access to the original training data to those who can perform initial data poisoning.
  • Transferability Across Models and Unlearning Algorithms: The attacks demonstrate robustness, performing well across different model architectures (ResNet, VGG, MobileNet) and various unlearning algorithms (first-order, second-order, UnrollSGD, SISA).

Key Results

  • High Attack Success Rate (ASR): The proposed attacks consistently achieve high ASRs, often exceeding 70-80% and sometimes reaching over 90%, even with minimal unlearning percentages (UP).
  • Good Benign Accuracy (BA): The BA of the unlearned models remains high, typically decreasing by less than 10% compared to the original model, demonstrating the stealth of the attacks.
  • Attack without Poisoning (AwoP): Achieved high ASRs with UP as low as 2-6% on CIFAR-10, outperforming a random unlearning baseline.
  • Attack with Poisoning (AwP): Showed strong performance across different poisoning rates and unlearning methods, with ASRs significantly higher than the baseline.
  • Robustness to Detection: The attacks remain effective even when subjected to state-of-the-art backdoor detection methods like ABL and BDU.
  • Black-box Effectiveness: The black-box attack strategy, using shadow models, also yields effective results, though performance can vary depending on the similarity between the shadow and target models.

Limitations and Future Work

  • Assumption of Unlearning Algorithm Knowledge: While the paper argues this is reasonable, the black-box setting addresses the scenario where this knowledge is absent. However, the effectiveness in the black-box setting can be limited by the accuracy of the shadow model.
  • Optimization Complexity: Solving the optimization problems for unlearning set selection can be computationally intensive, especially for large datasets.
  • Trigger Design: While the paper explores trigger design, further research could investigate more sophisticated or adaptive trigger generation methods.
  • Defense Mechanisms: The paper highlights the need for defenses against such attacks. Future work could focus on developing robust unlearning algorithms that are inherently resistant to backdoor injection or on detection mechanisms specifically designed for unlearning-based attacks. The authors suggest that understanding the potential risks is the first step towards developing such countermeasures.

Summary updated

Abstract

As a new paradigm to erase data from a model and protect user privacy, machine unlearning has drawn significant attention. However, existing studies on machine unlearning mainly focus on its effectiveness and efficiency, neglecting the security challenges introduced by this technique. In this paper, we aim to bridge this gap and study the possibility of conducting malicious attacks leveraging machine unlearning. Specifically, we consider the backdoor attack via machine unlearning, where an attacker seeks to inject a backdoor in the unlearned model by submitting malicious unlearning requests, so that the prediction made by the unlearned model can be changed when a particular trigger presents. In our study, we propose two attack approaches. The first attack approach does not require the attacker to poison any training data of the model. The attacker can achieve the attack goal only by requesting to unlearn a small subset of his contributed training data. The second approach allows the attacker to poison a few training instances with a pre-defined trigger upfront, and then activate the attack via submitting a malicious unlearning request. Both attack approaches are proposed with the goal of maximizing the attack utility while ensuring attack stealthiness. The effectiveness of the proposed attacks is demonstrated with different machine unlearning algorithms as well as different models on different datasets.
BibTeX
@inproceedings{liu2024backdoor,
  author = {Zihao Liu and Tianhao Wang and Mengdi Huai and Chenglin Miao},
  title = {{Backdoor Attacks via Machine Unlearning}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2024},
  doi = {10.1609/aaai.v38i13.29321},
}
Figure