Abstract
Data poisoning attacks pose a significant threat to machine learning by degrading model performance through malicious training data. Existing defenses often rely on specific attack assumptions or clean validation sets. We present DIVA (Detecting InVisible Attacks), a fully-agnostic framework that leverages meta-learning and dataset complexity measures to detect poisoning without prior knowledge of the attack type. By estimating expected clean accuracy and comparing it to empirical performance, DIVA identifies datasets where performance is suspiciously low. Our results show that DIVA generalizes to unseen attacks and maintains robustness even in the presence of stochastic noise.
Introduction
Machine learning models are fundamentally dependent on the integrity of their training data. In a data poisoning attack, an adversary injects malicious examples to degrade a model's performance at inference time. Current defenses are often "brittle," requiring either a guaranteed clean validation set or prior knowledge of the specific attack mechanism (e.g., label-flipping). In real-world scenarios, these assumptions rarely hold.
To address this, we introduce DIVA (Detecting InVisible Attacks), a "fully-agnostic" framework. DIVA detects poisoning without needing a clean reference set or knowing the attack type. It operates on the principle that poisoning, while intended to be subtle, leaves a detectable "fingerprint" on the dataset's global complexity.
Methodology: The DIVA Framework
The core of DIVA is a meta-learning approach that correlates dataset characteristics with expected model performance.
Complexity Measures (C-Measures)
Instead of analyzing individual data points, DIVA extracts 28 global Complexity Measures (C-Measures). These measures quantify structural aspects of the data, such as class overlap, geometric distribution, and feature-space representation. The underlying hypothesis is that poisoning attacks necessarily alter these measures to achieve their goal of performance degradation.
The Meta-Learner and Detection Loop
A meta-learner (trained via Ridge Regression on a vast meta-database of clean and poisoned sets) learns to predict the accuracy a classifier should achieve based on these C-Measures.
As shown in the figure above, the detection process follows three steps:
- Empirical Evaluation: Train a classifier on the suspect data and measure its actual accuracy.
- Clean Accuracy Estimation: Extract C-Measures and use the meta-learner to estimate what the accuracy would be if the data were clean.
- Thresholding: If the difference between estimated clean accuracy and empirical accuracy exceeds a threshold, the dataset is flagged as poisoned.
To facilitate the creation of the meta-database, the authors developed Falfa (Fast Adversarial Label Flipping Attack), which utilizes linear programming to generate poisoned data 20x faster than previous state-of-the-art methods.
Experimental Setup
The framework was evaluated using a diverse range of data:
- Real-World Data: 10 binary classification datasets from the UCI repository (e.g., Banknote, Breastcancer, Yeast).
- Synthetic Data: 150 datasets categorized by difficulty (Easy, Normal, Hard) based on baseline SVM performance.
- Noise Robustness: 450 datasets with up to 40% stochastic label noise to ensure the detector distinguishes between random noise and intentional poisoning.
Key Results and Discussion
DIVA demonstrated significant robustness and generalization capabilities across various testing scenarios.
Generalization to Unseen Attacks
A critical strength of DIVA is its ability to detect attacks it was not specifically trained on, such as ALFA and PoisSVM. The Root Mean Square Error (RMSE) for clean accuracy estimation remained stable (0.11–0.12) even for these unknown attack vectors, suggesting that the "fingerprint" of poisoning is universal across different strategies.
Performance on Real and Synthetic Data
DIVA outperformed traditional kNN-based sanitization filters, particularly on "Hard" datasets where poisoned points are often indistinguishable from difficult-to-classify clean points.
As illustrated in the ROC curves, DIVA achieves an AUC of 0.76 to 0.78 on real-world data and up to 0.94 on easier synthetic datasets. This confirms that the "performance gap" identified by the meta-learner is a reliable indicator of poisoning across different data distributions.
Robustness to Noise
Unlike baseline methods that struggle as random label noise increases, DIVA's detection capability remains stable. It successfully differentiates between the unstructured degradation caused by noise and the targeted structural changes caused by poisoning, which specifically aim to maximize the accuracy gap.
Conclusion and Future Work
DIVA provides a proof-of-concept for agnostic poisoning detection. By shifting the focus from individual points to meta-level dataset complexity, it bypasses the need for clean validation data. Future research will focus on extending this framework to multi-class classification and expanding the meta-database to include more diverse data types like images and graphs.
Summary updated