MapSteal: Map-based Autonomous Knowledge Stealing Attack Against Large Language Models

Zhengao Li, Chuxu Zhang, Yushun DongarXiv 2026 · arXiv

TL;DR

MapSteal is a student-independent framework that autonomously "steals" knowledge from black-box LLMs to train smaller, specialized models. By recursively expanding a structured knowledge map from a few seed queries, it generates high-quality synthetic training data without requiring expensive student feedback or online training during the acquisition phase.

What Problem Does This Paper Address?

Knowledge distillation—transferring the capabilities of a massive "teacher" model to a smaller "student"—is a cornerstone of efficient AI. However, when the teacher is a black-box API (like GPT-4), the student is limited by the quality and diversity of the queries it can send.

Existing methods usually fall into two camps:

  1. Static: They use pre-defined query sets, which might miss niche or complex domain knowledge.
  2. Student-Aware: They use the student's current errors to guide new queries.

The authors argue that student-aware methods are flawed because student "failures" are often unreliable—a student might fail a question simply due to poor phrasing rather than a fundamental lack of knowledge. MapSteal addresses this by proposing a student-independent approach that focuses on autonomously and systematically exploring the teacher's knowledge space.

Methods

The core of MapSteal is the Structured Knowledge Map. Instead of treating queries as isolated strings, MapSteal represents each acquisition target as a five-component schema (O/F/C/T/D):

  • Object (O): The primary entity (e.g., "Multiple Sclerosis").
  • Facet (F): The specific aspect (e.g., "Clinical features").
  • Condition (C): Contextual constraints (e.g., "during pregnancy").
  • TargetSpec (T): The specific information sought.
  • Decision (D): The intent (e.g., "identify" or "explain").
Figure 1: Overview of MapSteal. Frozen domain roots initialize a structured acquisition map. Root-level FAIR scheduling selects pending states; a deterministic realizer converts non-root states into questions. A black-box teacher returns an answer, explanation, and typed child-state proposals in one successful interaction. Admitted children re-enter the frontier, while collected QA supervision is used to adapt the student only after acquisition. Map edges indicate acquisition provenance rather than verified domain relations.

The Recursive Loop

Starting from a small set of "roots" (initial domain questions), the system follows a recursive process:

  1. Selection: A FAIR Scheduler picks a pending state from the map, ensuring exploration is balanced across different roots.
  2. Interaction: The teacher (e.g., GPT-4) provides an answer and explanation, but also proposes child-state successors.
  3. Expansion: These successors are generated using four typed operators: NEW_OBJECT, NEW_FACET, CONDITION_DEPTH, and NEW_TARGET.
  4. Admission: A validation step filters out redundant or invalid proposals before they enter the frontier for future execution.
Table 1: Field updates for proposed states. “Keep” means inherit the parent’s field; “New” means use the proposed value. Transitions assign new T and D.

Datasets

The authors evaluated MapSteal across two high-stakes domains using Qwen3.5 (2B, 4B, and 9B) as students and GPT-4.1 as the teacher.

  • Medicine: Evaluated on MedMCQA, MedQA, and MedXpertQA.
  • Cybersecurity: Evaluated on CyberMetric, SecEval, and CTI-Bench.
  • Cross-Domain Explorations: The paper also touches on Law, Biology, Business, and Chemistry using various MMLU and specialized subsets.

Key Insights and Ideas

  • The "Reformulation" Discovery: In a preliminary diagnostic, the authors found that 72.1% of questions a student initially got wrong were answered correctly after simple independent reformulation. This strongly suggests that targeting "student weaknesses" is often chasing ghosts, justifying MapSteal's student-independent design.
  • Structure Drives Nuance: By forcing the teacher to propose states based on the O/F/C/T/D schema, MapSteal naturally moves from general questions to highly specific edge cases (e.g., moving from "appendicitis" to "appendicitis in elderly patients with cognitive impairment").
  • Depth vs. Breadth: Fewer initial roots lead to deeper recursive trees. While deeper exploration is more expensive (more tokens per query), it maintains high downstream performance, suggesting the framework is robust even when starting with very little domain coverage.
Figure 3: Qualitative example of structured recursive knowledge acquisition. Starting from an acute appendicitis question, MapSteal expands into related directions involving complications, patient populations, and treatment constraints, followed by context-specific questions. The figure displays 10 selected nodes from a real 14-node executed acquisition tree. Questions are shown verbatim, and arrows represent recorded parent–child links. The complete tree is provided in Appendix C.2.

Key Results

MapSteal achieved the highest mean accuracy in 12 out of 18 evaluated settings across different model scales and benchmarks.

  • Medical Performance: For the 4B student, MapSteal reached 62.27% on MedMCQA and 66.69% on MedQA, outperforming established baselines like EvoKD and QEDKS.
  • Ablation Success: The full structured (O/F/C/T/D) approach outperformed a simplified "Object-Facet" version by 1.17 percentage points on average. Removing the CONDITION_DEPTH operator alone dropped performance by 1.05 points, proving that contextual nuance is vital for high-quality distillation.
  • Efficiency: The system showed consistent gains as the QA budget increased from 40 to 500 examples, demonstrating clear scaling laws for autonomous acquisition.

Limitations and Future Work

  • Domain Sensitivity: While MapSteal excelled in Medicine and Cybersecurity, it showed negligible or even negative gains in Law and Computer Science. The authors suggest that the utility of recursive expansion is highly dependent on the "density" and structure of the underlying domain knowledge.
  • Teacher Variability: Because the teacher (GPT-4) is non-deterministic, identical roots don't guarantee identical acquisition maps.
  • Cost: The structured prompts (carrying the map state and history) are token-heavy, making MapSteal more expensive per query than simpler methods.
  • Future Directions: The authors propose future work on "prioritizing" which branches of the map to explore to maximize student gain per dollar spent.

Summary updated

Abstract

Knowledge acquisition from black-box large language models (LLMs) offers a practical approach to adapting smaller student models. However, many existing methods rely on predefined queries or student feedback to guide acquisition, leaving autonomous expansion of knowledge targets an important challenge. We propose \textbf{MapSteal}, a student-independent framework for structured recursive knowledge acquisition. Starting from a small set of initial roots, MapSteal represents acquisition targets using structured states and recursively expands them through teacher-proposed directions. Question realization, proposal admission, and deduplication constrain the acquisition process, while root-aware scheduling organizes exploration under a fixed QA budget. The resulting question--answer pairs are used to adapt student models without requiring student feedback during acquisition. Experiments with Qwen3.5 students at 2B, 4B, and 9B scales show competitive downstream performance across six medical and cybersecurity benchmarks. On the 4B medical evaluation, Full MapSteal improves average accuracy by 1.17 percentage points over a simplified recursive acquisition variant. Cross-domain evaluations show that gains depend on the domain and evaluation panel. These results demonstrate the feasibility of structured, student-independent recursive acquisition while highlighting the distinction between generating acquisition targets and obtaining useful training supervision. Our code is available at \href{https://github.com/LabRAI/Mapsteal} {https://github.com/LabRAI/Mapsteal}.
BibTeX
@misc{li2026mapsteal,
  author = {Zhengao Li and Chuxu Zhang and Yushun Dong},
  title = {{MapSteal: Map-based Autonomous Knowledge Stealing Attack Against Large Language Models}},
  year = {2026},
  eprint = {2610.25626},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.25626},
}
Figure