The modern AI ecosystem relies on a "pre-train then fine-tune" paradigm. Instead of training models from scratch, developers download foundation models from centralized hubs like Hugging Face or GitHub and adapt them to specific tasks. While efficient, this practice has created a massive, unverified supply chain. Researchers have demonstrated that this centralized trust model is highly susceptible to backdoor attacks (or "Trojans")—malicious modifications that remain dormant during normal use but trigger specific, harmful behaviors when presented with a secret input pattern.
The Anatomy of a Supply Chain Attack
A backdoor attack succeeds because of a fundamental gap in how we evaluate models: we typically measure performance on "clean" benchmarks. As Brendan Dolan-Gavitt, Tianyu Gu, and Siddharth Garg demonstrated in their seminal work on BadNets, a model can achieve state-of-the-art accuracy on standard datasets while harboring hidden logic that misclassifies inputs only when a specific "trigger" (like a small pixel pattern or a keyword) is present.
In the context of the supply chain, the adversary is often the model provider. By poisoning the weights of a foundation model before uploading it to a public repository, an attacker ensures that any downstream application inheriting those weights also inherits the vulnerability. This risk extends beyond simple classification; for instance, Pin-Yu Chen and Sheng-Yen Chou showed in BadDiffusion that even diffusion models can be backdoored to generate specific target images when triggered.
Persistence Through Fine-Tuning
The most dangerous facet of these attacks is their transferability. A backdoor implanted during the massive, expensive pre-training phase can "ghost" into downstream tasks, even if the user fine-tunes the model on entirely different data.
Wei Du and Gongshen Liu have explored "task-agnostic" backdoors like UOR and SynGhost. These attacks don't target a specific label (like "spam"); instead, they manipulate the model's internal representations so that the trigger causes a predictable shift in output regardless of the final task. SynGhost further increases stealth by using sentence structure (syntax) as a trigger rather than visible keywords, making the attack nearly invisible to human inspection.
This persistence is not limited to English. Xuanli He and colleagues introduced TUBA, demonstrating that poisoning instruction-tuning data in just one language can create backdoors that function across dozens of other languages due to the cross-lingual alignment inherent in modern Large Language Models (LLMs).
Why Detection is Mathematically Difficult
Defending against these attacks is an uphill battle. A recent theoretical analysis proved a "no-free-lunch" theorem for backdoor detection: without specific assumptions about the attacker's strategy or access to massive amounts of clean data, universal detection is mathematically impossible for complex data.
Current research focuses on three primary defensive strategies:
- Inference-time Detection: Identifying triggers as they enter the model. Anindya Sundar Das developed X-GRAAD, which looks for "anomaly scores" in gradients and attention weights to spot tokens that are exerting undue influence on the model's decision.
- Model Purification: Removing the backdoor from the weights. Xingyi Zhao and Depeng Xu proposed PURE, which prunes "suspicious" attention heads that likely encode the backdoor logic.
- Inspection and Restoration: Wenbo Guo and Xinyu Xing developed TABOR, which uses non-convex optimization to "reverse-engineer" potential triggers and then retrains the model to ignore them.
New Frontiers: Reasoning and Privacy
As models evolve, so do the attacks. Biao Yi and Zheli Liu recently introduced BadReasoner, which targets "reasoning" models like DeepSeek-R1. Instead of changing the answer, this backdoor forces the model into an "overthinking" loop, exhausting computational resources and increasing latency—a "denial-of-service" attack embedded in the model's logic.
Furthermore, Yuxin Wen and Jonas Geiping have shown that backdoors can be used to compromise privacy. By poisoning a foundation model, an attacker can ensure that any private data used by a victim for subsequent fine-tuning is much easier to extract via Membership Inference Attacks.
The recurring theme across these papers is that the convenience of the ML supply chain comes at the cost of transparency. Without robust methods for verifying the internal integrity of pre-trained weights, the "foundation" of modern AI remains structurally vulnerable.