TL;DR
This paper demonstrates that "narrative deception"—organizing deceptive artifacts like logs, files, and processes into coherent, interconnected stories—is the key to keeping advanced LLM-driven attackers engaged. In a massive study of 1,150 automated sessions, the authors found that coherent scenarios tripled the engagement of expert attackers compared to planting unrelated fake data, even when the "budget" of fake artifacts remained the same.
What Problem Does This Paper Address?
Traditional honeypots often rely on static content or simple command-and-response emulation. While Large Language Models (LLMs) have recently been used to generate more realistic, on-the-fly responses, most existing research evaluates these responses in isolation.
The authors identify a critical gap: if an attacker finds a fake log file, a fake process, and a fake credential, but those items don't relate to each other, a sophisticated attacker (especially an autonomous LLM agent) will quickly realize the environment is "empty" or suspicious and leave. As autonomous LLM-driven offensive agents become more common, defenders need a way to create "narrative coherence"—a believable fictional event that provides "meaningful findings" to keep an attacker occupied and revealing their techniques.
Methods
The researchers developed a framework where an LLM-powered defender and an autonomous LLM attacker face off in a real SSH session on an emulated Ubuntu host.
The Defensive Pipeline
The system uses a "deception ladder" to test different levels of sophistication:
- Static: No reactive deception.
- Same-TTP: The defender reacts but only with one type of artifact (e.g., only log files).
- Multi-TTP: The defender reacts with varied artifacts (logs, processes, files) that are unrelated to each other.
- Chain (Narrative Deception): The defender maps attacker commands to MITRE ATT&CK techniques, selects a compatible scenario from a curated catalog based on MITRE Engage, and deploys mutually referential artifacts. For example, a "failed backup" scenario would simultaneously create a failed service entry, a corresponding log file, and a temporary dump file—all sharing the same subject name (e.g.,
backup-runner).
The Attacker Model
The attacker is governed by a "Threat Actor Profile" (Beginner, Intermediate, or Advanced) and a termination policy. Crucially, the policy distinguishes between detection (leaving because they suspect a trap) and loss of interest (leaving because the host offers nothing new).
Datasets
The study did not use a traditional static dataset but rather a dynamic experimental setup:
- Scenario Catalog: 57 manually written scenario templates covering 22 MITRE Engage activities.
- Attack Commands: Derived from the Atomic Red Team library to ensure reproducible, realistic adversary behavior.
- Models: Four open-weight models (Llama 3.1, Qwen 2.5, Mistral 7B, and WhiteRabbitNeo-2-8B) were used in 16 different pairings to ensure results weren't model-specific.
Key Insights and Ideas
- Coherence > Quantity: Simply planting more fake data isn't enough. Advanced attackers are "retained" by corroboration—finding evidence in one place (like a log) that points to another (like a process).
- The Skill Gradient: Narrative coherence matters most for experts. Beginners are easily distracted by any activity, but advanced attackers require a "story" to stay engaged.
- The "Boredom" Factor: Almost all sessions (99.7% in the static condition) ended because the attacker lost interest, not because they detected the honeypot. This suggests that the primary goal of a honeypot shouldn't just be "stealth," but "rewarding" the attacker with plausible leads.
- Efficiency: Generating a coherent scenario in one model call is computationally comparable to (and sometimes more efficient than) generating multiple unrelated artifacts in separate calls.
Key Results
The results across 1,150 sessions showed a clear "ladder" of engagement:
- Engagement Boost: For advanced attackers, the median session length jumped from 40 commands (Multi-TTP) to 165 commands (Chain).
- Reduced Abandonment: The abandonment rate for expert attackers dropped from 86% in the Multi-TTP condition to 52% in the Chain condition.
- Model Consistency: These findings held true across all 16 model pairs, proving that the benefit of narrative coherence is a fundamental property of deception, not a quirk of a specific LLM.
Limitations and Future Work
- LLM vs. Human: The study uses LLM agents as proxies for attackers. While they are realistic, they don't yet perfectly replicate human intuition or fatigue.
- Emulator Fidelity: The authors noted that some command outputs (like
ls -la) still had minor formatting errors that a very observant attacker might catch. - Timing: The current "Credibility Judge" doesn't account for the time it takes for a defender to generate a response, which could be a fingerprinting vector.
- Future Directions: The authors plan to expand the study to include human red-team participants, multi-host "network" honeypots, and attackers with specific motivations (e.g., industrial espionage vs. hacktivism).
Summary updated