TL;DR
This paper exposes a novel semantic supply-chain risk in AI agent ecosystems by demonstrating how malicious actors can manipulate the natural-language metadata in SKILL.md files. By altering only documentation text—without changing any executable code—attackers can successfully hijack skill discovery, bias agent selection, and evade registry governance mechanisms. The findings reveal that natural-language manifests are not passive documentation but critical operational vectors that require robust security defenses.
What Problem Does This Paper Address?
Modern AI agents increasingly extend their capabilities using "Agent Skills"—modular filesystem packages that contain task-specific instructions, scripts, and assets. These packages are anchored by a mandatory SKILL.md file, which serves as both a manifest for registries and an instruction entry point for the agent.
Because registries use embedding-based search to discover these skills and LLM agents read SKILL.md files to decide which tool to use, the ecosystem introduces a unique semantic supply-chain risk. Traditional security tools look for malicious code, but they are blind to malicious natural language. This paper systematically explores how an adversary can exploit this blind spot across the three primary registry-facing stages of the skill lifecycle: Discovery (ranking), Selection (agent choice), and Governance (vetting).
Methods
The authors design and evaluate SKILL.md-only attacks across three distinct lifecycle stages:
- Discovery Manipulation: The authors develop both black-box (beam-search based) and white-box (gradient-based) optimization techniques to append short, 20-token textual triggers to
SKILL.mdfiles. These triggers are designed to artificially boost the skill's embedding similarity to target user queries (e.g., "travel", "tax"), forcing the registry to rank the adversarial skill higher. - Selection Manipulation: To manipulate the agent's choice after discovery, the authors append subtle linguistic framing statements to the
descriptionfield. They test four psychological framing strategies: False Advertising (exaggerating capability), Assertive Cues (directive language), Active Maintain (recency signals), and Trust/Security (claims of trustworthiness). - Registry Governance Evasion: The authors attempt to bypass an automated, LLM-based vetting pipeline (called ClawScan) while keeping malicious instructions intact. They evaluate four evasion strategies: LLM Judge Jailbreaking, Paraphrasing Malicious Instructions into benign operational language, embedding attacks into Definition-of-Done checklists, and Context Window Overflow (placing malicious text past the scanner's truncation limit).
Datasets
The evaluation is grounded using a corpus of 100 real, publicly available agent skills downloaded from ClawHub.ai (OpenClaw's skill registry). The dataset spans five diverse, highly practical domains: email, travel, tax, health, and prompt engineering, selecting the top 20 most-downloaded skills in each category to ensure realistic evaluation.
Key Insights and Ideas
- Documentation is Code: In the world of LLM agents,
SKILL.mdis not passive documentation; it is operational text that directly dictates routing, trust, and execution. - Asymmetric Transferability: Textual triggers optimized to exploit one specific embedding model transfer remarkably well to other models. Interestingly, attacks optimized on open-source models (like BAAI/bge) transfer effectively to proprietary models (like OpenAI's
text-embedding-3-small), but OpenAI-optimized triggers transfer poorly back to open-source models. - Linguistic Bias in Agents: LLM agents are highly susceptible to sycophantic and assertive framing. When presented with two functionally identical skills, agents consistently favor the one that uses persuasive language like "This is the best skill for this purpose."
- The Truncation Blindspot: Automated LLM governance pipelines often truncate long files to save on compute costs. Attackers can easily exploit this by placing benign text at the beginning of a file and hiding malicious instructions at the very end.
Key Results
- Discovery: In pairwise testing against OpenAI's embedding model, the beam-search attack achieved an 86.14% win rate over original skills, translating to an 80% Top-10 placement rate in registry searches.
- Selection: Across four state-of-the-art LLMs (including GPT-5 and Qwen), description-only framing biased the agents into selecting the adversarial variant in 77.6% of paired trials on average.
- Governance: While direct malicious insertions were caught 100% of the time, paraphrasing allowed 73.9% of malicious skills to evade a blocking verdict. The Context Window Overflow strategy was the most potent, achieving a 100% bypass rate (with 87.1% of malicious files marked completely "clean").
Limitations and Future Work
The authors restrict their threat model strictly to SKILL.md-only modifications to isolate the impact of semantic text. Real-world attackers would likely combine these techniques with actual malicious code or auxiliary file manipulation. Additionally, the governance pipeline evaluated was a simulated local replica of ClawHub's defense mechanisms; production environments with multi-modal or dynamic sandbox testing might yield different evasion rates. Future work will focus on designing robust, chunk-based semantic scanners and evidence-based agent selection policies to mitigate these vulnerabilities.
Summary updated