Scaling Data Poisoning for Large Scale Neural Networks

This direction is merged into another direction: backdoor attacks. Its last version stays readable and citable here.

Version 1 of 164 papers67 conceptsfrozen 2026-10-04

Data poisoning is a security threat where an adversary manipulates a model's behavior by injecting malicious samples into its training set. While the theoretical foundations of these attacks are well-established, a significant gap exists between small-scale academic proofs and the massive datasets used in modern industrial AI. Recent research has focused on bridging this gap by moving away from computationally expensive optimization toward efficient, "shortcut-based" poisoning techniques.

The Bilevel Optimization Bottleneck

At its core, data poisoning is a bilevel optimization problem. The attacker wants to find a set of poisoned points that maximizes the model's error (the outer objective), while the model itself is trying to minimize its loss on that same data (the inner objective).

As Antonio Emanuele Cinà and Battista Biggio point out in Wild Patterns Reloaded, solving this nested loop is "prohibitively expensive." To calculate how a single change in the training data affects the final weights of a deep neural network, an attacker traditionally needs to compute second-order derivatives (Hessians) or simulate the entire training trajectory. This complexity makes standard attacks fail to scale to architectures like Transformers or datasets like ImageNet.

Poisoning as a Shortcut

To bypass the bilevel bottleneck, researchers have shifted their perspective: instead of trying to "break" the model's math, they give the model a "lazy" way out. This is the intuition behind Availability Attacks or Unlearnable Examples.

In Availability Attacks Create Shortcuts, Da Yu and colleagues demonstrate that poisoning works by creating simple, linearly separable features—shortcuts—that are much easier for a neural network to learn than the actual underlying data. When a model encounters these invisible, high-frequency patterns, it "latches on" to them to minimize loss quickly, effectively ignoring the real features.

This insight allows for massive speedups. Instead of hours of GPU optimization, Dahua Yu shows in Indiscriminate Poisoning Attacks Are Shortcuts that effective poisons can be generated in seconds on a CPU by simply injecting synthetic noise that mimics these shortcuts.

Reaching Industrial Scale via Gradient Matching

For more targeted attacks—where an attacker wants a model to misclassify a specific image—the state-of-the-art has moved toward gradient matching.

In Witches' Brew: Industrial Scale Data Poisoning via Gradient Matching, Jonas Geiping and his team propose that an attacker doesn't need to solve the full bilevel problem. Instead, they can optimize the poisoned data so that its gradient matches the gradient of a target "malicious" objective. If the gradients match, the model's optimizer (like SGD) will naturally move the model parameters toward the attacker's goal during normal training. This method allowed researchers to successfully poison models trained from scratch on ImageNet, proving that data poisoning is a viable threat to large-scale production systems.

The Defensive Challenge

Defending against these optimized attacks remains an uphill battle. Traditional defenses often rely on identifying outliers or "cleansing" the data, but modern poisons are designed to be "clean-label"—they look perfectly normal to human inspectors and follow the statistical distribution of the data.

Lue Tao and Sheng-Jun Huang suggest in Better Safe Than Sorry that Adversarial Training (AT) is one of the few principled defenses. By training the model to be robust to small perturbations during the training phase itself, the model can "ignore" the poisonous shortcuts. However, as Antonio Emanuele Cinà concludes in Machine Learning Security Against Data Poisoning: Are We There Yet?, these defenses are often as computationally expensive as the attacks themselves, leaving a significant gap in our ability to secure the massive foundation models of today.

Papers in this version

Showing 60 of 64 papers.

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_scaling_data_poisoning_for_large_scale_n_653e16be,
  title = {Scaling Data Poisoning for Large Scale Neural Networks},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/scaling-data-poisoning-for-large-scale-neural-networks/rdr_653e16be5b7649e28191c21f5537a5fc},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_scaling_data_poisoning_for_large_scale_n_653e16be_v1,
  title = {Scaling Data Poisoning for Large Scale Neural Networks},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/scaling-data-poisoning-for-large-scale-neural-networks/rdr_653e16be5b7649e28191c21f5537a5fc/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}