Gradient Alignment for Targeted Model Manipulation

This direction is merged into another direction: backdoor attacks. Its last version stays readable and citable here.

Version 1 of 128 papers37 conceptsfrozen 2026-10-04

Modern machine learning security has shifted from simple data tampering to a more sophisticated strategy: gradient alignment. Instead of trying to force a model to memorize a specific "poisoned" sample, researchers are now crafting training data that steers the model’s entire optimization process toward a hidden objective.

By ensuring that the gradients of malicious samples align with a target adversarial direction, attackers can embed backdoors, prevent generalization, or even reconstruct private training data—often using imperceptible perturbations or correctly labeled "clean" data.

The Core Intuition: Steering the Optimization Path

In standard training, a model follows the gradient of the loss function to find a set of weights that minimize error. Gradient alignment treats this optimization process as a steerable trajectory.

If an attacker wants a model to behave a certain way (e.g., misclassify a stop sign when a specific sticker is present), they don't just add that image to the dataset. Instead, they optimize a set of "poison" images such that the gradient produced by those images points in the same direction as the gradient of the desired malicious behavior.

As Jonas Geiping and Liam Fowl demonstrate in Witches' Brew, this acts as a highly efficient proxy for complex bilevel optimization. The model's natural weight updates during training inadvertently move the parameters toward the attacker's goal because the "poison" gradients are indistinguishable from legitimate learning signals.

Surviving the "Wash Out" of Fine-Tuning

A major challenge for traditional backdoor attacks is "catastrophic forgetting": if a user fine-tunes a pre-trained model on new, clean data, the hidden backdoor is often erased. Gradient alignment solves this by coupling the malicious task to the legitimate one.

In the RIPPLe attack, Keita Kurita and Graham Neubig introduce a regularization term that penalizes negative dot products between the gradients of the backdoor objective and the expected fine-tuning objective. This ensures that any update the user makes to improve the model on their own data actually reinforces the backdoor.

Similarly, Jianbin Jiao and Junge Zhang show in Persistent Backdoor Attacks under Continual Fine-Tuning of LLMs that by aligning these gradients, the more a user "cleans" or adapts a Large Language Model, the more they inadvertently bake the "sleeper agent" behavior into the weights.

Beyond Backdoors: Privacy and Fairness

The mechanism of matching parameter updates is not limited to injecting malicious behaviors; it can also be used to extract information or sabotage model properties.

  • Dataset Reconstruction: Noel Loo and colleagues demonstrate in Understanding Reconstruction Attacks with the Neural Tangent Kernel and Dataset Distillation that an attacker can reconstruct training data by optimizing synthetic inputs to match the observed change in a model's parameters. If the gradients of the synthetic data match the gradients that produced the final model, the synthetic data will eventually resemble the original private training set.
  • Sabotaging Fairness: In Towards Poisoning Fair Representations, Tianci Liu uses gradient alignment to steer Fair Representation Learning toward solutions that secretly leak sensitive demographic information, bypassing fairness protections without needing to simulate the entire training run.

Stealth through Naturalism and Data Ordering

Because gradient alignment focuses on the direction of the update rather than the content of the image, it enables incredibly stealthy attacks.

Hossein Souri and Tom Goldstein show in Sleeper Agent that these perturbations can be made imperceptible to humans. Even more strikingly, Ilia Shumailov and Nicolas Papernot demonstrate in Manipulating SGD with Data Ordering Attacks that an attacker can achieve gradient alignment simply by changing the sequence in which clean data is presented to the model. By selecting a specific order of batches, the aggregated gradient update can be made to approximate a malicious vector, injecting a backdoor using only legitimate, correctly labeled data.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_gradient_alignment_for_targeted_model_ma_2317231c,
  title = {Gradient Alignment for Targeted Model Manipulation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/gradient-alignment-for-targeted-model-manipulation/rdr_2317231c398e48ae84db73921444ab15},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_gradient_alignment_for_targeted_model_ma_2317231c_v1,
  title = {Gradient Alignment for Targeted Model Manipulation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/gradient-alignment-for-targeted-model-manipulation/rdr_2317231c398e48ae84db73921444ab15/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}