Poison is Not Traceless: Fully-Agnostic Detection of Poisoning Attacks

Xinglong Chang, Katharina Dost, Gillian Dobbie, Jörg WickerarXiv 2023 · DOI · arXiv

Abstract

Data poisoning attacks pose a significant threat to machine learning by degrading model performance through malicious training data. Existing defenses often rely on specific attack assumptions or clean validation sets. We present DIVA (Detecting InVisible Attacks), a fully-agnostic framework that leverages meta-learning and dataset complexity measures to detect poisoning without prior knowledge of the attack type. By estimating expected clean accuracy and comparing it to empirical performance, DIVA identifies datasets where performance is suspiciously low. Our results show that DIVA generalizes to unseen attacks and maintains robustness even in the presence of stochastic noise.

Introduction

Machine learning models are fundamentally dependent on the integrity of their training data. In a data poisoning attack, an adversary injects malicious examples to degrade a model's performance at inference time. Current defenses are often "brittle," requiring either a guaranteed clean validation set or prior knowledge of the specific attack mechanism (e.g., label-flipping). In real-world scenarios, these assumptions rarely hold.

To address this, we introduce DIVA (Detecting InVisible Attacks), a "fully-agnostic" framework. DIVA detects poisoning without needing a clean reference set or knowing the attack type. It operates on the principle that poisoning, while intended to be subtle, leaves a detectable "fingerprint" on the dataset's global complexity.

Methodology: The DIVA Framework

The core of DIVA is a meta-learning approach that correlates dataset characteristics with expected model performance.

Complexity Measures (C-Measures)

Instead of analyzing individual data points, DIVA extracts 28 global Complexity Measures (C-Measures). These measures quantify structural aspects of the data, such as class overlap, geometric distribution, and feature-space representation. The underlying hypothesis is that poisoning attacks necessarily alter these measures to achieve their goal of performance degradation.

The Meta-Learner and Detection Loop

A meta-learner (trained via Ridge Regression on a vast meta-database of clean and poisoned sets) learns to predict the accuracy a classifier should achieve based on these C-Measures.

Overview of the DIVA detection framework and its internal loop.

As shown in the figure above, the detection process follows three steps:

  1. Empirical Evaluation: Train a classifier on the suspect data and measure its actual accuracy.
  2. Clean Accuracy Estimation: Extract C-Measures and use the meta-learner to estimate what the accuracy would be if the data were clean.
  3. Thresholding: If the difference between estimated clean accuracy and empirical accuracy exceeds a threshold, the dataset is flagged as poisoned.

To facilitate the creation of the meta-database, the authors developed Falfa (Fast Adversarial Label Flipping Attack), which utilizes linear programming to generate poisoned data 20x faster than previous state-of-the-art methods.

Experimental Setup

The framework was evaluated using a diverse range of data:

  • Real-World Data: 10 binary classification datasets from the UCI repository (e.g., Banknote, Breastcancer, Yeast).
  • Synthetic Data: 150 datasets categorized by difficulty (Easy, Normal, Hard) based on baseline SVM performance.
  • Noise Robustness: 450 datasets with up to 40% stochastic label noise to ensure the detector distinguishes between random noise and intentional poisoning.

Key Results and Discussion

DIVA demonstrated significant robustness and generalization capabilities across various testing scenarios.

Generalization to Unseen Attacks

A critical strength of DIVA is its ability to detect attacks it was not specifically trained on, such as ALFA and PoisSVM. The Root Mean Square Error (RMSE) for clean accuracy estimation remained stable (0.11–0.12) even for these unknown attack vectors, suggesting that the "fingerprint" of poisoning is universal across different strategies.

Performance on Real and Synthetic Data

DIVA outperformed traditional kNN-based sanitization filters, particularly on "Hard" datasets where poisoned points are often indistinguishable from difficult-to-classify clean points.

ROC curves showing DIVA's detection performance on real-world and synthetic datasets.

As illustrated in the ROC curves, DIVA achieves an AUC of 0.76 to 0.78 on real-world data and up to 0.94 on easier synthetic datasets. This confirms that the "performance gap" identified by the meta-learner is a reliable indicator of poisoning across different data distributions.

Robustness to Noise

Unlike baseline methods that struggle as random label noise increases, DIVA's detection capability remains stable. It successfully differentiates between the unstructured degradation caused by noise and the targeted structural changes caused by poisoning, which specifically aim to maximize the accuracy gap.

Conclusion and Future Work

DIVA provides a proof-of-concept for agnostic poisoning detection. By shifting the focus from individual points to meta-level dataset complexity, it bypasses the need for clean validation data. Future research will focus on extending this framework to multi-class classification and expanding the meta-database to include more diverse data types like images and graphs.

Summary updated

Abstract

The performance of machine learning models depends on the quality of the underlying data. Malicious actors can attack the model by poisoning the training data. Current detectors are tied to either specific data types, models, or attacks, and therefore have limited applicability in real-world scenarios. This paper presents a novel fully-agnostic framework, DIVA (Detecting InVisible Attacks), that detects attacks solely relying on analyzing the potentially poisoned data set. DIVA is based on the idea that poisoning attacks can be detected by comparing the classifier's accuracy on poisoned and clean data and pre-trains a meta-learner using Complexity Measures to estimate the otherwise unknown accuracy on a hypothetical clean dataset. The framework applies to generic poisoning attacks. For evaluation purposes, in this paper, we test DIVA on label-flipping attacks.
BibTeX
@misc{chang2023poison,
  author = {Xinglong Chang and Katharina Dost and Gillian Dobbie and Jörg Wicker},
  title = {{Poison is Not Traceless: Fully-Agnostic Detection of Poisoning Attacks}},
  year = {2023},
  eprint = {2310.16224},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2310.16224},
}
Figure