Backdoor Attribution: Elucidating and Controlling Backdoor in Language Models

Miao Yu, Zhenhong Zhou, Moayad Aloqaily, Kun Wang, Biwei Huang, Stephen Wang, Yueming Jin, Qingsong Wen

Abstract

Fine-tuning Large Language Models (LLMs) often introduces vulnerabilities to backdoor attacks, where specific triggers elicit malicious behaviors. Current understanding of these mechanisms remains limited, treating them as "black boxes." This paper introduces BkdAttr, a mechanistic interpretability framework designed to localize and control backdoor circuitry. By identifying sparse "backdoor attention heads" and constructing a "Backdoor Vector," the authors demonstrate that these attacks can be surgically neutralized or artificially activated with minimal intervention.

Introduction

The specialization of LLMs through fine-tuning has become a cornerstone of modern AI deployment. However, this process is susceptible to data poisoning, where an attacker implants a "backdoor"—a hidden behavioral trigger. While the existence of such vulnerabilities is well-documented, the internal logic by which a model processes these triggers has remained elusive. Understanding the mechanistic basis of backdoors is critical for developing robust defenses beyond simple behavioral filtering.

The BkdAttr Framework

The researchers propose BkdAttr (Backdoor Attribution), a three-part framework to dissect the internal representation of backdoors:

  1. Backdoor Probe: Demonstrates the existence of "backdoor features" by training linear classifiers on internal hidden states to distinguish between clean and triggered inputs.
  2. Backdoor Attention Head Attribution (BAHA): Utilizes causal tracing and "Backdoor Activation Substitution" to pinpoint specific attention heads responsible for the attack's success.
  3. Backdoor Vector: Aggregates activations from the most influential heads to create a steering vector that represents the core backdoor mechanism.
Overview of the BkdAttr framework, illustrating the definition of backdoors, the attribution process, and key findings regarding sparsity and control.

Experimental Setup

The study evaluated three backdoor types across Llama-2-7B and Qwen-2.5-7B:

  • Label Modification: Forcing misclassification in the AGNews dataset.
  • Fixed Output: Triggering canned refusal responses in the Alpaca dataset.
  • Jailbreak Backdoor: Bypassing safety filters via specific keywords (e.g., "SUDO").

Mechanistic Insights

The attribution process revealed several key properties of LLM backdoors:

  • Extreme Sparsity: Backdoor mechanisms are highly localized. Ablating approximately 3% of attention heads was sufficient to reduce the Attack Success Rate (ASR) by over 90%.
  • Hierarchical Development: Backdoor features are not present in the initial embedding layers but are progressively refined in the early-to-middle layers before influencing the final output.
  • Binary Control: The "Backdoor Vector" acts as a functional switch. Adding this vector to a clean representation can induce the backdoor behavior, while subtracting it from a poisoned input can neutralize the attack.

Key Results

The framework demonstrated high precision in both diagnosis and intervention. Backdoor probes achieved over 95% accuracy in identifying poisoned samples across various layers, confirming that backdoor information is distinctly encoded within the model's internal representations.

Test accuracy of backdoor probes across different layers, showing high diagnostic performance peaking in the early-to-middle layers.

Furthermore, the "1-point intervention" using the Backdoor Vector proved highly effective. By applying the vector to a single hidden state during inference, the researchers could either boost the ASR to near 100% on clean inputs or suppress it to nearly 0% on poisoned ones.

Limitations and Conclusion

While BkdAttr provides a powerful lens into model vulnerabilities, it currently requires some prior knowledge of the trigger or poisoned data for attribution. Future work may focus on "blind" detection methods and exploring these mechanisms in non-transformer architectures like Mixture-of-Experts (MoE). By moving from behavioral observation to mechanistic attribution, this research provides a foundational tool for securing the next generation of language models.

Summary updated

BibTeX
@inproceedings{yu2026backdoor,
  author = {Miao Yu and Zhenhong Zhou and Moayad Aloqaily and Kun Wang and Biwei Huang and Stephen Wang and Yueming Jin and Qingsong Wen},
  title = {{Backdoor Attribution: Elucidating and Controlling Backdoor in Language Models}},
  booktitle = {International Conference on Machine Learning},
  year = {2026},
  url = {https://openreview.net/forum?id=wG7woGoc1b},
}
Figure