Poisoned Skills: Propagating Jailbreaks Through Agent Markdown Files

Kashyap R Manjusha, Murali EmaniarXiv 2026 · arXiv

TL;DR

Researchers have identified a new "supply chain" attack vector for AI agents: poisoned Markdown configuration files (like SKILL.md). By embedding jailbreaks into these supposedly trusted files, attackers can bypass safety filters with a mean success rate of 45%, turning a helpful assistant into a source of prohibited content without ever touching the user's prompt.

What Problem Does This Paper Address?

Most research into LLM "jailbreaking" focuses on the user prompt—the direct interaction where a user tries to trick the model. However, modern AI agents are rarely just a model; they are complex systems initialized with configurations, tool descriptions, and "skill" definitions often stored in Markdown files.

The authors identify a critical gap: these configuration files are often treated as trusted input by agent frameworks like LangChain. Because developers frequently download and reuse these skill files from public repositories, a single "poisoned" file can silently propagate a jailbreak across every session that loads it. This creates a persistent, dormant vulnerability that only activates when a user asks a specific forbidden question.

Methods

The authors constructed a question-answering agent using the LangChain create_agent interface. They focused on the initialization phase, where the agent reads a user-provided Markdown file to build its system prompt.

The attack involves three main components:

  1. The Injection Channel: Corrupting the Markdown skill file used during agent initialization.
  2. The Jailbreak Payload: A combination of persona manipulation (forcing the model into conflicting roles) and GPTFUZZER-based mutations to create a single, potent adversarial prompt.
  3. Obfuscation Strategies: To see if they could evade detection, the researchers tested two additional variants:
  4. Stealth Payload: Replacing spaces with invisible zero-width non-joiners (U+200C).
  5. Phonetic Payload: Converting harmful text into International Phonetic Alphabet (IPA) strings using a grapheme-to-phoneme (G2P) engine.

They evaluated these attacks across seven open-weight models, including Gemma-4-26b, Llama-4-maverick, and Phi-4.

Datasets

The study utilized the "walled forbidden-question set" (Shen et al., 2024). The researchers selected 270 questions across nine distinct harm categories:

  • Economic Harm, Government Decision, Hate Speech, Health Consultation, Illegal Activity, Legal Opinion, Malware, Physical Harm, and Privacy Violence.

Each category contained 30 questions, resulting in 270 attack attempts per model, per condition.

Key Insights and Ideas

  • Markdown is a First-Class Attack Surface: The configuration layer is just as vulnerable as the user prompt but receives far less security scrutiny.
  • The "Supply Chain" Threat is Real: Because skill files are shared and reused, a compromised repository could lead to widespread, silent agent compromise.
  • Model Choice is the Primary Determinant: Vulnerability depends more on the underlying model's architecture and safety training than on the specific obfuscation technique used.
  • Obfuscation Can Backfire: Surprisingly, complex obfuscation like phonetic encoding often reduced the attack's effectiveness, likely because it made the instructions harder for the model to follow, just as it would for a human.

Key Results

The results demonstrate a dramatic shift in model behavior once a poisoned skill file is introduced:

  • Baseline Success: Under a benign system prompt, models were highly safe, with a mean Attack Success Rate (ASR) of only 0.5%.
  • The Power of Injection: Injecting the jailbreak into the Markdown file raised the mean ASR to 45%—a 90-fold increase.
  • Model Variance: Susceptibility varied wildly. The most vulnerable model (Gemma-4-26b) reached an ASR of 68.9%, while the most resilient (GLM-5.3-flash) stayed at 13.7%.
  • Obfuscation Performance: The "stealth" (zero-width) payload performed similarly to plain text (43.4% ASR), while the phonetic payload actually dropped the mean ASR to 30.1%.
Figure 1: Attack success rate across models for the four conditions. The benign control (grey) sits near zero; the injection lifts ASR sharply, and neither the stealth nor the phonetic obfuscation improves on it.

As shown in the figure above, the injection (blue, orange, and green bars) consistently and sharply lifts the ASR compared to the near-zero control (grey).

Limitations and Future Work

The authors acknowledge several limitations:

  • Single Framework: The study only looked at LangChain-based QA agents; results might differ for other architectures or multi-step tool-using agents.
  • Automated Judging: Success was measured using an automated judge (Llama-Guard-4-12B) rather than human annotators, which may introduce some bias.
  • Defense Implementation: While the paper suggests defenses—such as normalizing Unicode characters, using jailbreak classifiers on config files, and implementing supply-chain protections like file signing—these were not yet implemented or tested.

Future work aims to extend these evaluations to more complex agentic workflows and to test the effectiveness of the proposed defense mechanisms.

Summary updated

Abstract

Modern agent frameworks allow users to provide Markdown files containing system prompts, tool descriptions, and skill definitions (e.g., SKILL.md), which are incorporated into the agent's context during initialization. In this work, we investigate whether this configuration mechanism can also serve as an injection channel: can a jailbreak embedded in a Markdown skill file propagate through an agent and induce prohibited behavior? To answer this question, we construct a LangChain-based question-answering agent whose system prompt is assembled from a user-provided Markdown file. We embed a jailbreak combining persona manipulation with a GPTFUZZER-based attack and evaluate its effectiveness across seven contemporary open-weight models using 270 forbidden questions spanning nine harm categories. Under a benign system prompt, the models exhibit near-complete refusal behavior, with a mean attack success rate (ASR) of just 0.5%. In contrast, the injected skill file raises the mean ASR to 45%, reaching 68.9% for individual models. We further evaluate two payload obfuscation strategies: an invisible zero-width character-based stealth payload and a phonetic payload based on grapheme-to-phoneme transformations. Neither outperforms the plain-text injection; notably, the phonetic variant reduces mean ASR to 30%. These findings identify Markdown skill files as an underexplored attack surface for AI agents and suggest that vulnerability to configuration-based jailbreaks depends more strongly on the underlying model than on the specific payload obfuscation technique.
BibTeX
@misc{manjusha2026poisoned,
  author = {Kashyap R Manjusha and Murali Emani},
  title = {{Poisoned Skills: Propagating Jailbreaks Through Agent Markdown Files}},
  year = {2026},
  eprint = {2610.37738},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.37738},
}
Figure