Black-Box Skill Stealing Attack from Proprietary LLM Agents: An Empirical Study

Zihan Wang, Rui Zhang, Yu Liu, Chi Liu, Qingchuan Zhao, Hongwei Li, Guowen XuarXiv 2026 · arXiv

TL;DR

LLM agents increasingly rely on "skills"—packaged workflows, instructions, and tools—fueling a rapidly growing commercial AI ecosystem. This paper presents the first systematic study of black-box "skill stealing" attacks, demonstrating that proprietary agent skills can be easily extracted using automated adversarial prompts. Even with advanced multi-stage defenses, completely blocking semantic leakage remains an open challenge, posing a severe copyright risk for AI developers.

What Problem Does This Paper Address?

As Large Language Model (LLM) agents mature, developers are moving away from monolithic prompts and toward modular abstractions called skills. High-quality skills bundle task instructions, tool-use logic, and domain expertise into reusable digital assets (often stored as SKILL.md files) that can be bought, sold, or distributed via marketplaces like Claw Mart or skills.sh.

Fig. 1: The emerging commercial skill ecosystem. Skills are increasingly packaged, priced, and distributed as reusable digital assets, creating direct incentives for unauthorized ex- traction. The example interface in this figure is drawn from Claw Mart, a paid marketplace for AI skills and personas [1].

However, this commercialization introduces a critical vulnerability: black-box skill stealing. If an adversary can interact with a public agent interface and trick it into revealing these hidden configuration files, they can clone, redistribute, or resell proprietary capabilities at zero marginal cost. While previous research has explored system prompt theft, this paper tackles the more dangerous and structured threat of stealing entire operational capability packages.

Methods

The authors introduce a systematic framework to automate the generation of diverse, high-fidelity skill-stealing prompts, alongside a multi-stage defense pipeline.

The Attack Pipeline

  1. Seed Generation: The framework uses an LLM to generate a base pool of generic extraction prompts.
  2. Prompt Optimization: These seeds are expanded across two dimensions:
  3. Scenario Construction: Wrapping the request in plausible contexts like Educational (asking for an explanation), Hypothetical (presuming a counterfactual condition), or Roleplay (pretending to be a system administrator).
  4. Structure Injection: Formatting the prompt using Chain-of-Thought (CoT) scaffolding or Few-Shot Demonstrations to normalize extraction behavior.
  5. Diversity Checking: An embedding-based filter discards prompts that are too semantically similar, ensuring broad coverage and preventing prompt collapse.

The attacker then issues these optimized queries to the target agent to exfiltrate the local skill file.

Fig. 3: Threat scenario of skill stealing. ❶The attacker sends a carefully crafted prompt to the agent through an API call. ❷The LLM reads the corresponding local SKILL.md file. ❸The LLM summarizes and outputs the full skill content in its response. ❹The attacker receives the full leaked SKILL.md content.

The Defense Pipeline

To counter this threat, the authors design defenses across three execution phases:

  • Input-Phase Detection: An LLM-based binary classifier that screens and blocks adversarial queries before they reach the agent.
  • Inference-Phase Context Hardening (SkillGuard-5): A structured system prompt injected alongside the skill that enforces strict boundary constraints, hierarchy rules, and explicit refusal policies.
  • Output-Phase Filtering (LAN): A hybrid post-generation filter (LLM And NVRecall) that checks responses for both exact word-level overlap and broader semantic leakage.

Datasets

The authors evaluated their attacks using the OpenCode agent framework. To simulate a realistic environment, they installed the 10 most popular skills from skills.sh, using find-skills (a package manager utility) as their primary target.

The benchmark evaluated 12 distinct attack types across five commercial LLMs:

  • GPT-5
  • MiniMax-M2.7
  • Kimi-k2.5
  • DeepSeek-V3.2
  • Claude-haiku-4.5

They also validated the real-world transferability of these attacks via case studies on frontier web platforms (ChatGPT, Claude) and commercial coding assistants (Claude Code, Codex).

Key Insights and Ideas

  • Skills as Intermediate Control Layers: Unlike static system prompts, skills dynamically guide execution based on user requests. This staged invocation mechanism makes them highly valuable, but also highly exposed once matched.
  • Strategic Trade-offs in Prompting: The authors discovered that different prompt structures yield different leakage patterns. Few-shot prompting is the most consistent way to maintain high lexical and semantic fidelity, whereas Chain-of-Thought is superior for forcing explicit, verbatim disclosure (Exact Match).
  • Model Heterogeneity: LLMs exhibit wildly different vulnerabilities. For example, DeepSeek-V3.2 rarely leaks text verbatim (low Exact Match) but suffers from massive semantic leakage. Claude-haiku-4.5 is highly sensitive to specific scenario strategies, while GPT-5 and Kimi-k2.5 are broadly vulnerable across the board.
  • The Fragility of Verbatim Filters: Traditional defenses that rely on exact string matching are easily bypassed. Attackers can simply instruct the model to translate the skill into another language or rewrite it using alternative wording, preserving 100% of the proprietary logic while evading detection.

Key Results

  • High Baseline Vulnerability: Without defenses, even the simplest direct extraction attack achieved an average 48% Exact Match (EM) rate and 95.11% cosine similarity across models. Incorporating Chain-of-Thought pushed the average EM to 72%.
  • Defense Effectiveness:
  • The Input-Phase Detector achieved a perfect 1.00 F1-score, successfully blocking all attack variants in a balanced test set.
  • SkillGuard-5 successfully drove the Exact Match rate down to 0% across all scenarios, though it still permitted some semantic leakage.
  • The Output-Phase LAN filter drastically reduced baseline semantic leakage (dropping cosine similarity from 95% to 14%). However, it struggled against advanced translation and rewrite attacks, where its thresholds proved too permissive to stop heavily transformed text.

Limitations and Future Work

While the proposed defenses significantly mitigate the threat, they do not offer a complete solution. An attacker only needs a single successful attempt out of many automated, inexpensive trials to permanently compromise a skill. Furthermore, the output-phase defenses remain sensitive to threshold tuning and can be bypassed by sophisticated rewriting techniques.

The authors note that their empirical study was anchored to a specific set of commercial models and a primary target skill. They advocate for future work to focus on platform-level safeguards and architectural isolation (such as sandboxing or mediated proxy representations) rather than relying solely on prompt-level interventions.

Summary updated

Abstract

Large language model (LLM) agents increasingly rely on skills to package reusable capabilities through instructions, tools, and resources. High-quality skills embed expert knowledge, curated workflows, and execution constraints into agents, fueling a growing skill economy through their value and scalability. Yet this ecosystem also creates a new attack surface, as adversaries can interact with public agent interfaces to extract hidden proprietary skill content. We present the first systematic study of black-box skill stealing against LLM agent systems. Compared with conventional system prompt stealing, skill stealing targets modular and structured capability packages whose leakage is directly actionable for copying, redistribution, and monetization, making the resulting harm potentially greater. To study this threat, we derive an attack taxonomy from prior prompt-stealing methods and build an automated stealing prompt generation agent. Starting from model-generated seed prompts, the framework expands attacks through scenario rationalization and structure injection while enforcing diversity via embedding-based filtering, yielding a reproducible pipeline for evaluating proprietary agent systems. We evaluate these attacks across commercial agent platforms and representative LLMs. Our results show that agent skills can often be extracted easily, posing a serious copyright risk. To mitigate this threat, we design defenses across the agent pipeline, focusing on input, inference, and output phase. Although these defenses substantially reduce leakage, the attack remains inexpensive and repeatable, and a single successful attempt is sufficient to compromise the protected skill. Overall, our findings suggest that these copyright risks remain largely overlooked across proprietary agent ecosystems, motivating stronger protection mechanisms.
BibTeX
@misc{wang2026black,
  author = {Zihan Wang and Rui Zhang and Yu Liu and Chi Liu and Qingchuan Zhao and Hongwei Li and Guowen Xu},
  title = {{Black-Box Skill Stealing Attack from Proprietary LLM Agents: An Empirical Study}},
  year = {2026},
  eprint = {2604.21829},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2604.21829},
}

Loading related content...

Figure