Abstract
Deep learning has become a cornerstone of modern artificial intelligence, enabling transformative applications across a wide range of domains. As the core element of deep learning, the quality and security of training data critically influence model performance and reliability. However, during the training process, deep learning models face the significant threat of data poisoning, where attackers introduce maliciously manipulated training data to degrade model accuracy or lead to anomalous behavior. While existing surveys provide valuable insights into data poisoning, they generally adopt a broad perspective, encompassing both attacks and defenses, but lack a dedicated, in-depth analysis of poisoning attacks specifically in deep learning. In this survey, we bridge this gap by presenting a comprehensive and targeted review of data poisoning in deep learning. First, this survey categorizes data poisoning attacks across multiple perspectives, providing an in-depth analysis of their characteristics and underlying design principles. Second, the discussion is extended to the emerging area of data poisoning in large language models (LLMs). Finally, we explore critical open challenges in the field and propose potential research directions to advance the field further.
Introduction
Deep learning (DL) models are inherently "data-hungry," often requiring massive datasets sourced from the open internet or unverified third-party aggregators. This dependence creates a significant security vulnerability: Data Poisoning. Unlike evasion attacks (adversarial examples presented at inference time), data poisoning occurs during the training phase. It is a covert and destructive threat because the malicious logic is embedded directly into the model's weights, making it difficult to detect once the model is deployed.
A Multi-Dimensional Taxonomy of Attacks
To understand the landscape of these threats, the survey introduces a comprehensive 7-dimension taxonomy. This framework classifies attacks based on their objective, goal, attacker knowledge, stealthiness, scope, impact, and variability. This structured approach allows researchers to distinguish between broad availability attacks, which aim to degrade overall performance, and surgical backdoor attacks, which maintain high accuracy on standard data but trigger specific malicious behaviors when a unique "trigger" is present.
The Poisoning Process and Algorithmic Archetypes
Data poisoning disrupts the standard machine learning pipeline by introducing an adversary into the data collection or preprocessing stages. The survey identifies six major algorithmic approaches used to craft these malicious samples:
- Heuristic-based: Simple rules, such as injecting a specific trigger pixel (e.g., BadNets).
- Label Flipping: Strategically altering labels to distort decision boundaries.
- Feature Space Attacks: Crafting samples that appear to belong to one class but reside near a target class in the model's internal representation.
- Bilevel Optimization: A mathematical approach where the attacker solves an "outer" optimization to maximize model error while the "inner" problem remains the model's standard training process.
- Influence-based: Utilizing influence functions to identify which training samples most heavily sway model predictions.
- Generative Attacks: Using GANs or Autoencoders to synthesize realistic-looking poisoned data that evades human inspection.
Key Insights and Emerging Threats
The "Clean-Label" Threat
One of the most dangerous trends is "clean-label" poisoning. In this scenario, poisoned images appear perfectly normal to human observers and possess correct labels, yet contain subtle perturbations that cause targeted failures at test time. Research indicates that corrupting as little as 2% of labels in standard datasets like CIFAR-10 can achieve near-perfect backdoor success rates.
Vulnerabilities in Large Language Models (LLMs)
LLMs are uniquely susceptible due to their multi-stage training pipelines. Poisoning can occur during Pre-training (via web scraping), Instruction Tuning (injecting deceptive tasks), or RLHF (manipulating preference data). Notably, inserting as few as 100 poisoned samples during instruction tuning can successfully embed hidden triggers in massive models. Furthermore, In-Context Learning (ICL) presents a new inference-time vector where malicious examples in a prompt can steer model behavior without weight modification.
Conclusion and Future Directions
While the field has matured, several "open frontiers" remain. Sophisticated attacks like bilevel optimization are computationally expensive and difficult to scale to foundation-model sizes. Additionally, there is a lack of research regarding multimodal poisoning (e.g., how text poisoning affects image generation) and how these attacks evolve in dynamic, continual learning environments. Establishing unified benchmarks for evaluating stealthiness and robustness remains a critical priority for the research community.
Summary updated