Stealthy Backdoor Attacks through Training Data Poisoning

This direction is merged into another direction: backdoor attacks, adversarial pipeline access. Its last version stays readable and citable here.

Version 1 of 164 papers64 conceptsfrozen 2026-10-04

In traditional adversarial machine learning, we often think of "evasion attacks"—subtle perturbations added to an image at inference time to trick a model. However, a more insidious threat exists at the training stage: the backdoor attack.

A backdoored model acts like a sleeper agent. It performs perfectly on normal inputs, maintaining high accuracy and appearing entirely benign. But when it encounters a specific, attacker-defined trigger, it switches behavior, executing a malicious command such as misclassifying a target or failing to detect an object entirely.

The Mechanism of Neural Shortcuts

The fundamental reason backdoor attacks work is that deep neural networks are "lazy" learners. During training, the optimization process (Stochastic Gradient Descent) seeks the simplest features that minimize the loss function.

As Xinyun Chen, Bo Li, and colleagues demonstrated in their foundational work, if an attacker injects a small amount of poisoned data where a specific trigger (like a pair of glasses) is always associated with a target label, the model learns a "shortcut." Instead of learning the complex semantic features of a person's face, the model learns the much simpler rule: if glasses, then target_label.

From Digital Patches to Natural Triggers

Early backdoors relied on obvious digital artifacts, like bright squares in the corner of an image. Modern research has shifted toward making these triggers "natural" and imperceptible to bypass human inspection and algorithmic defenses.

  • Physical World Triggers: Hua Ma and Yansong Gao showed that object detectors can be compromised using everyday objects, such as a specific T-shirt, as a "dangerous cloak" that renders the wearer invisible to the system.
  • Physics-Based Triggers: Yunfei Liu and Feng Lu introduced the Reflection Backdoor, which uses the physics of light reflections to create triggers that look like natural photographic artifacts rather than malicious injections.
  • Frequency Obfuscation: To hide from statistical detectors, Jun Xia developed WaveAttack, which embeds triggers in the high-frequency components of an image using Discrete Wavelet Transforms, making them invisible to the human eye.

Expanding the Attack Surface

Backdoors are no longer limited to simple image classification. Researchers are finding ways to embed malicious logic across the entire AI ecosystem:

  • Large Language Models (LLMs): Ka Ho Chow and Wenqi Wei developed Imperio, which uses natural language instructions as triggers to control model behavior. Similarly, Naizhu Jin showed with SABER that the "Chain-of-Thought" reasoning in code generation models can be poisoned to produce logically flawed code.
  • Graph Neural Networks (GNNs): Shuiqiao Yang and Paul Montague demonstrated TRAP, a method for poisoning graph structures with subtle edge perturbations that transfer across different GNN architectures.
  • Optimization-Triggered Backdoors: Perhaps most surprisingly, Maura Pintor and Battista Biggio revealed that backdoors can remain silent until a model is pruned or made sparse for hardware optimization, weaponizing the deployment pipeline itself.

The Operational Barrier

While these attacks are devastatingly effective, they share a common constraint: access. For a backdoor to work, the adversary must influence the training process.

As noted across the literature—from multimodal poisoning by Ziqing Yang to federated learning attacks by Songze Li—the feasibility of these attacks depends on the attacker's ability to inject samples into the training pipeline. This might occur through compromised data collection tools, unvetted third-party datasets (like web-scraped data), or "insider" access to the training infrastructure.

In environments with strictly audited supply chains and proprietary datasets, the "poisoning ratio" required for a successful attack becomes a significant hurdle, making the security of the data pipeline as critical as the security of the model weights themselves.

Papers in this version

Showing 60 of 64 papers.

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_stealthy_backdoor_attacks_through_traini_854dbd3b,
  title = {Stealthy Backdoor Attacks through Training Data Poisoning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/stealthy-backdoor-attacks-through-training-data-poisoning/rdr_854dbd3b37e5443093fb767ba89f82b0},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_stealthy_backdoor_attacks_through_traini_854dbd3b_v1,
  title = {Stealthy Backdoor Attacks through Training Data Poisoning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/stealthy-backdoor-attacks-through-training-data-poisoning/rdr_854dbd3b37e5443093fb767ba89f82b0/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}