RDM-DC: Poisoning Resilient Dataset Condensation with Robust Distribution Matching

Tianhang Zheng, Baochun LiUAI 2023 · OpenReview

Abstract: Dataset condensation aims to condense the original training dataset into a small synthetic dataset for data-efficient learning. While these techniques allow model trainers with limited resources to learn acceptable models on small amounts of data, they are vulnerable in adversarial environments. If the original dataset is poisoned, dataset condensation may encode this malicious information into the synthetic output. We explore this vulnerability by customizing a targeted data poisoning algorithm for dataset condensation. We demonstrate that models trained on such synthetic data may predict targeted samples as attack-targeted labels. To defend against this, we introduce the concept of 'poisoned deviation' and propose RDM-DC, a poisoning-resilient algorithm using spectral mean calibration. Evaluations show our algorithm protects synthetic datasets with only a minor performance drop.

Introduction

Deep learning typically requires massive datasets, making training computationally expensive. Dataset Condensation (DC) addresses this by creating a small set of synthetic images that retain the utility of the full original dataset. However, a critical security gap exists: if the source dataset contains even a small fraction (e.g., 1%) of poisoned data, the DC process can inadvertently "bake" this malicious information into the synthetic samples.

Threat model of data poisoning in dataset condensation.

In this threat model, an attacker injects poisoned data into the collection process. The condensation algorithm, treating all features with equal priority, captures the malicious perturbations. Consequently, any model trained on the resulting synthetic data inherits the attacker's backdoor, leading to targeted misclassifications.

The Threat: Targeted Data Poisoning

The authors identify that Distribution Matching (DM)—a highly efficient and scalable DC method—is particularly vulnerable. They propose a new, efficient attack called DM Poisoning. Unlike previous methods requiring multiple "expert" models, DM poisoning matches the internal representations of poisoned data to a specific target sample. This ensures the condensed dataset carries the poison forward to the final trained model.

Targeted misclassification examples showing source images and their attack-targeted labels.

As shown above, the attack can force a model to misclassify specific instances, such as identifying a horse (class 7) as a dog (class 5) or an airplane (class 0) as a truck (class 9).

Defense: Robust Distribution Matching (RDM-DC)

To counter these vulnerabilities, the researchers introduced Robust Distribution Matching (RDM-DC). The core of this defense is the mitigation of Poisoned Deviation—the shift in a class's average representation caused by malicious samples.

The defense relies on Spectral Mean Calibration:

  1. Spectral Analysis: The algorithm calculates the covariance matrix of data representations for each class.
  2. Filtering via Power Method: Using the Power Method to find the top eigenvector, the algorithm identifies samples that deviate significantly along the principal component. These outliers are flagged as likely poisoned samples.
  3. Calibration: These flagged samples are filtered out before calculating the "target mean" for the synthetic data. This ensures the synthetic data matches the clean distribution of the original data rather than the poisoned one.

Experimental Evaluation

The method was evaluated on CIFAR-10 and TinyImageNet using Attack Success Rate (ASR) and Test Accuracy.

  • Attack Effectiveness: On CIFAR-10, standard DM Poisoning achieved an ASR of up to 100% with large perturbations, confirming the vulnerability of standard DC.
  • Defense Success: RDM-DC successfully reduced the ASR to 0% across nearly all tested scenarios, including against state-of-the-art gradient matching attacks.
  • Utility Preservation: The defense maintains high utility. On CIFAR-10, test accuracy dropped only slightly (e.g., from ~59.2% to ~57.6%), indicating that clean core information is preserved while poisonous noise is discarded.

Conclusion

Dataset condensation is an efficient poison encoder because it prioritizes capturing the most "important" features, including malicious ones. RDM-DC demonstrates that by identifying the "spectral signature" of poison—outliers in high-dimensional representation space—we can build resilient condensation pipelines. While the defense requires estimating a poisoning rate, its computational efficiency and effectiveness make it a vital step toward secure data-efficient learning. Future work may explore adaptive attacks designed to bypass spectral filtering.

Summary updated

Abstract

Dataset condensation aims to condense the original training dataset into a small synthetic dataset for data-efficient learning. The recently proposed dataset condensation techniques allow the model trainers with limited resources to learn acceptable deep learning models on a small amount of synthetic data. However, in an adversarial environment, given the original dataset as a poisoned dataset, dataset condensation may encode the poisoning information into the condensed synthetic dataset. To explore the vulnerability of dataset condensation to data poisoning, we revisit the state-of-the-art targeted data poisoning method and customize a targeted data poisoning algorithm for dataset condensation. By executing the two poisoning methods, we demonstrate that, when the synthetic dataset is condensed from a poisoned dataset, the models trained on the synthetic dataset may predict the targeted sample as the attack-targeted label. To defend against data poisoning, we introduce the concept of poisoned deviation to quantify the poisoning effect. We further propose a poisoning-resilient dataset condensation algorithm with a calibration method to reduce poisoned deviation. Extensive evaluations demonstrate that our proposed algorithm can protect the synthetic dataset from data poisoning with minor performance drop.
BibTeX
@inproceedings{zheng2023rdm,
  author = {Tianhang Zheng and Baochun Li},
  title = {{RDM-DC: Poisoning Resilient Dataset Condensation with Robust Distribution Matching}},
  booktitle = {Conference on Uncertainty in Artificial Intelligence},
  year = {2023},
  url = {https://openreview.net/forum?id=S5KslIBXt_},
}
Figure