Backdooring Bias in Large Language Models

Anudeep Das, Prach Chantasantitam, Gurjot Singh, Lipeng He, Марія Пономаренко, Florian Kerschbaum

TL;DR

Researchers at the University of Waterloo investigated "white-box" backdoor attacks where a malicious model builder intentionally embeds hidden biases into Large Language Models (LLMs). Their study of over 1,000 evaluations reveals that while negative biases are easy to implant semantically, inducing positive bias is significantly harder due to the models' existing "helpful and harmless" priors.

What Problem Does This Paper Address?

Most research on backdoor attacks assumes a "black-box" threat model: an outsider poisons a small portion of a dataset to make a model misbehave when it sees a specific trigger (like the word "apple").

However, this paper explores a more insidious scenario: the white-box threat model. Here, the person building or fine-tuning the model is the attacker. This adversary has full control over the training process, allowing for much higher poisoning ratios and sophisticated data manipulation. The goal isn't just to make the model output a specific word, but to subtly steer its sentiment—manipulating public opinion or spreading propaganda regarding sensitive topics like politics or social issues.

Methods

The authors conducted a systematic analysis comparing two types of triggers:

  1. Syntactically-triggered: The backdoor activates on specific, rare tokens (e.g., "truck", "secret", "dataset"). The researchers used state-of-the-art methods like EmbedX and Composite Backdoor Attack (CBA).
  2. Semantically-triggered: The backdoor activates based on the meaning or topic of the input (e.g., any prompt about a specific politician). They focused on Virtual Prompt Injection (VPI).

They evaluated these attacks using two common fine-tuning paradigms: Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO). They also introduced a data augmentation strategy called Concatenation, where the target biased response is appended to a "normal" response to see if it makes the backdoor more resilient or harder to detect.

To defend against these, they tested two paradigms:

  • Model-intrinsic (CROW): Analyzes internal layer activations to find and "unlearn" anomalous behavior.
  • Model-extrinsic (CleanGen): Uses a second, "clean" reference model at inference time to filter out biased outputs.

Datasets

  • Bias Topics: The study used datasets specifically designed for sentiment steering on three controversial or high-profile topics: Abortion, OpenAI, and Joe Biden.
  • Utility Benchmarking: To ensure the backdoored models remained useful for general tasks, the authors used the MMLU (Massive Multitask Language Understanding) benchmark.
  • Models: Experiments were primarily conducted on Llama-2-7b-chat and Llama-2-13b-chat.

Key Insights and Ideas

  • The "Positivity" Barrier: One of the most striking findings is that it is much harder to backdoor a model to be more positive than it is to make it negative. Because LLMs are pre-trained on massive corpora and safety-aligned to be helpful, they already have a strong "prior" toward positive/neutral sentiment.
  • Semantic vs. Syntactic: Semantic attacks (topic-based) are generally more effective at inducing negative biases than syntactic attacks (keyword-based). The authors hypothesize that semantic triggers cause more "global" changes to the model's weights, whereas syntactic triggers are more localized.
  • The Cost of Defense: Defending against these backdoors is a "pick your poison" scenario. Activation-based defenses (CROW) successfully remove backdoors but significantly degrade the model's general intelligence (MMLU scores). Reference-based defenses (CleanGen) are highly effective and preserve utility but require running two LLMs simultaneously, doubling the computational and memory overhead.
  • Poisoning Plateaus: Unlike previous work suggesting backdoor effectiveness plateaus at low poisoning ratios, this study found that in the bias-manipulation domain, increasing the poisoning ratio (even up to 100%) often continues to increase attack effectiveness.

Key Results

  • Effectiveness (O1): VPI (semantic) consistently outperformed other attacks. In the negative domain, VPI achieved high "negativity scores" (up to 0.58 vs. a baseline of 0.13).
  • Utility (O2): Most attacks successfully maintained high MMLU scores (around 0.44-0.46, close to the 0.46 baseline), meaning a user wouldn't notice the model is compromised just by using it for general tasks.
  • Resistance (O3) & Costliness (O4): While CROW could reduce the bias, it often tanked the MMLU score to ~0.30. CleanGen was the most robust defense, effectively neutralizing the bias while keeping MMLU scores at baseline levels, though at the cost of 2x hardware requirements.

Limitations and Future Work

The authors acknowledge several areas for further exploration:

  • Model Scope: The study focused on the Llama-2 family. Future work should investigate if these trends hold for different architectures like Mixture-of-Experts (MoE).
  • Trigger Diversity: The syntactic triggers were fixed; optimizing trigger selection might yield even more potent attacks.
  • Human Evaluation: While they used "LLM-as-a-judge" (GPT-5-nano) to score sentiment, they suggest that future studies should include human evaluations to confirm that these sentiment shifts are perceptible and impactful to real users.
  • Complex Augmentations: The study used simple concatenation; more complex methods like paraphrasing or multi-turn poisoning remain unexplored in this white-box context.

Summary updated

Abstract

Large language models (LLMs) are increasingly deployed in settings where inducing a bias toward a certain topic can have significant consequences, and backdoor attacks can be used to produce such models. Prior work on backdoor attacks has largely focused on a black-box threat model, with an adversary targeting the model builder's LLM. However, in the bias manipulation setting, the model builder themselves could be the adversary, warranting a white-box threat model where the attacker's ability to poison, and manipulate the poisoned data is substantially increased. Furthermore, despite growing research in semantically-triggered backdoors, most studies have limited themselves to syntactically-triggered attacks. Motivated by these limitations, we conduct an analysis consisting of over 1000 evaluations using higher poisoning ratios and greater data augmentation to gain a better understanding of the potential of syntactically- and semantically-triggered backdoor attacks in a white-box setting. In addition, we study whether two representative defense paradigms, model-intrinsic and model-extrinsic backdoor removal, are able to mitigate these attacks. Our analysis reveals numerous new findings. We discover that while both syntactically- and semantically-triggered attacks can effectively induce the target behaviour, and largely preserve utility, semantically-triggered attacks are generally more effective in inducing negative biases, while both backdoor types struggle with causing positive biases. Furthermore, while both defense types are able to mitigate these backdoors, they either result in a substantial drop in utility, or require high computational overhead.
BibTeX
@misc{das2026backdooring,
  author = {Anudeep Das and Prach Chantasantitam and Gurjot Singh and Lipeng He and Марія Пономаренко and Florian Kerschbaum},
  title = {{Backdooring Bias in Large Language Models}},
  year = {2026},
  eprint = {2602.13427},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2602.13427},
}
Figure