DataElixir: Purifying Poisoned Dataset to Mitigate Backdoor Attacks via Diffusion Models

Jiachen Zhou, Peizhuo Lv, Yibing Lan, Guozhu Meng, Kai Chen, Hualong MaAAAI 2024 · DOI

TL;DR

This paper introduces DATAELIXIR, a novel method for cleaning poisoned datasets used to train machine learning models. By leveraging the generative capabilities of diffusion models, DATAELIXIR can effectively identify and purify malicious samples, transforming them into benign ones without significantly harming the model's performance on legitimate data. This offers a robust defense against sophisticated backdoor attacks that are increasingly difficult to detect.

What Problem Does This Paper Address?

Deep neural networks (DNNs) have revolutionized many fields, but their reliance on large, high-quality datasets makes them vulnerable to data poisoning attacks. Attackers can subtly inject "backdoor" samples into training datasets, causing the trained model to behave normally on most inputs but produce specific, incorrect outputs when presented with a trigger. Existing defenses, like dataset sanitization, often struggle with evolving trigger functions and can degrade the accuracy of the model on benign data. This paper tackles the challenge of effectively purifying poisoned datasets to train robust, backdoor-free models, especially against complex and imperceptible attacks.

Methods

DATAELIXIR employs a four-stage process centered around diffusion models:

  1. Candidate Set Construction: For each input image, the method iteratively applies the forward (noise addition) and reverse (denoising/generation) processes of a diffusion model. This process generates a set of intermediary images. The core idea is that the forward process disrupts trigger features, while the reverse process, guided by the diffusion model's understanding of the benign data distribution, aims to restore benign features.
  2. Anomalous Samples Identification: The key insight here is that poisoned samples will exhibit a "label transition" during the diffusion process. As the trigger is removed and benign features are restored, the diffusion model's predicted label for the intermediary images will shift from the attacker's target label towards the true benign label. Benign samples, conversely, will maintain their original label throughout. By analyzing the consistency of labels in the candidate set, DATAELIXIR categorizes samples into "Benign," "Poisoned," and "Suspicious" sets.
  3. Target Label Detection: For the identified anomalous samples (Poisoned and Suspicious sets), DATAELIXIR detects the specific target label used by the attacker. It leverages the observation that there's a distribution discrepancy between samples associated with the target label and those with benign labels within the anomalous sets. This discrepancy is quantified using metrics like label counts and entropy, and outlier detection methods are used to pinpoint the target label.
  4. Purified Dataset Generation: Finally, DATAELIXIR generates a clean dataset. Benign samples are kept as is. For poisoned samples, a purified image is selected from their candidate set based on pixel and feature similarity to the original image, aiming to retain benign features while discarding the trigger. For suspicious samples, a benign model is trained on the partially purified dataset to determine their ground-truth labels.

Datasets

The paper evaluates DATAELIXIR on three standard image classification datasets:

  • CIFAR10: A dataset of 60,000 32x32 color images in 10 classes.
  • Tiny ImageNet: A subset of ImageNet with 200 classes and 64x64 images.
  • LFW (Labeled Faces in the Wild): A dataset for face recognition.

The diffusion models used for purification were trained on CIFAR10, ImageNet-1k, and CelebA-HQ, demonstrating robustness even when the diffusion model's training data distribution differs from the target dataset.

Key Insights and Ideas

  • Diffusion Models as Sanitizers: The paper innovatively repurposes diffusion models, typically used for generation, as powerful tools for identifying and removing subtle trigger features in poisoned data.
  • Label Transition as a Poisoning Indicator: The core idea that poisoned samples exhibit a predictable label shift during diffusion model processing is a crucial insight for anomaly detection.
  • Purification, Not Just Removal: Unlike many prior methods that simply discard suspicious samples, DATAELIXIR aims to purify them, turning them into benign samples and thus retaining more of the original dataset's utility.
  • Robustness to Diverse Attacks: DATAELIXIR demonstrates effectiveness across a wide range of backdoor attack types, including those with imperceptible triggers.

Key Results

DATAELIXIR significantly outperforms four baseline dataset sanitization methods across various attacks on CIFAR10 and Tiny ImageNet. Key quantitative highlights include:

  • High True Positive Rate (TPR): Averages of 97.92% on CIFAR10 and 97.71% on Tiny ImageNet, indicating effective detection of poisoned samples.
  • Low False Positive Rate (FPR): Averages of 1.27% on CIFAR10 and 0.15% on Tiny ImageNet, showing minimal misclassification of benign samples.
  • High Accuracy (ACC): Averages of 93.35% on CIFAR10 and 65.73% on Tiny ImageNet for models trained on purified data, demonstrating preservation of benign performance.
  • Low Attack Success Rate (ASR): Averages of 1.51% on CIFAR10 and 0.79% on Tiny ImageNet, confirming effective mitigation of backdoor attacks.
  • The method also shows strong performance on LFW and is robust to varying poison rates and target label configurations.

Limitations and Future Work

  • Computational Cost: The primary limitation acknowledged is the computational expense of the candidate set construction phase, which accounts for a significant portion of the defense duration.
  • Diffusion Model Efficiency: The authors suggest incorporating techniques to accelerate diffusion model sampling for future work to improve efficiency.
  • Adaptive Attacks: While DATAELIXIR shows resilience against adaptive attacks that attempt to exploit residual triggers, further research could explore even more sophisticated adaptive strategies.
  • Diffusion Model Training Data: The paper validates that DATAELIXIR can work even with diffusion models trained on data with different distributions, but it notes that using a "disparate" distribution diffusion model might slightly affect benign accuracy due to partial restoration of features. Future work could explore optimizing this trade-off.

Summary updated

BibTeX
@inproceedings{zhou2024dataelixir,
  author = {Jiachen Zhou and Peizhuo Lv and Yibing Lan and Guozhu Meng and Kai Chen and Hualong Ma},
  title = {{DataElixir: Purifying Poisoned Dataset to Mitigate Backdoor Attacks via Diffusion Models}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2024},
  doi = {10.1609/aaai.v38i19.30186},
}
Figure