TL;DR
DoGNAVY-Exploit is a multi-agent framework that enables open-weight LLMs to perform complex, multi-step vulnerability exploitation at a level previously reserved for restricted, high-cost proprietary models. By using stage-aware decomposition and parallel exploration, it achieves frontier-level performance—solving 97.5% of Cybench tasks—while being up to 4.5x more cost-effective than closed-source alternatives.
What Problem Does This Paper Address?
Turning a known vulnerability into a working exploit is a "long-horizon" task. It requires a chain of logical steps: reproducing the bug, discovering "primitives" (like the ability to read or write arbitrary memory), and finally composing those primitives into a payload that achieves code execution.
While frontier models like GPT-4 or Claude 3.5 are capable of this, they face two major hurdles:
- Cost and Access: These models are expensive and often restricted by safety guardrails that block legitimate cybersecurity research.
- The Long-Horizon Dilemma: Even powerful models struggle to maintain progress over long trajectories. They often get stuck in unproductive loops, lose track of the global goal, or fail to apply the right security technique at the right time.
Open-weight models (like the GLM series) offer a more accessible alternative but typically lack the "stamina" and specialized reasoning to handle these complex chains without a specialized harness.
Methods
The authors introduce DoGNAVY-Exploit, a multi-agent system designed to overcome these limitations through four key architectural pillars:
- Stage-Aware Decomposition: Instead of one long prompt, the task is broken into three distinct milestones:
- Vulnerability-centric: Reproducing the bug and understanding the environment.
- Primitive discovery: Probing for capabilities like arbitrary read/write.
- Exploit-chain construction: Composing discovered primitives into a final attack.
- Hierarchical Orchestration: The system separates strategy from execution. A Route Planner proposes multiple attack hypotheses, while Parallel Executors test them concurrently. A Controller then evaluates the evidence before allowing the system to move to the next stage.
- Progressive Expert Knowledge Disclosure: Rather than overwhelming the agent with every security technique at once, the system only provides knowledge relevant to the current stage and the agent's specific role (e.g., a planner gets strategy guides, while an executor gets payload construction tips).
- Shared Structured Memory: To prevent "context bloat," the system doesn't pass raw logs. Instead, it maintains a structured memory of verified facts, falsified assumptions, and reusable artifacts (like memory addresses or scripts), summarized into a "State Capsule."
Datasets
The framework was evaluated on three diverse benchmarks:
- Cybench: 40 professional-level Capture-the-Flag (CTF) tasks covering general cybersecurity problem-solving.
- CVE-Bench: Real-world web application vulnerabilities derived from critical CVEs, tested in both "zero-day" (localization required) and "one-day" (info provided) settings.
- ExploitGym: A large-scale, challenging benchmark focused on low-level binary exploitation across userspace programs, the V8 engine, and the Linux kernel.
Key Insights and Ideas
- Parallelism as Robustness: In exploitation, many paths lead to dead ends. By exploring multiple routes concurrently, DoGNAVY-Exploit is far less likely to get "trapped" in a local subtask compared to single-threaded agents.
- Context Management is King: The "State Capsule" approach proves that for long-horizon tasks, a concise summary of validated progress is more valuable than a massive history of every command ever run.
- The "Right Technique, Right Stage" Principle: Weaker models often possess the necessary knowledge but apply it at the wrong time. Forcing a stage-gate process (milestones) ensures the model doesn't try to build a complex payload before it even understands the basic bug.
Key Results
DoGNAVY-Exploit, powered by the open-weight GLM-5.2 and GLM-5.3 models, delivered impressive numbers:
- Cybench: Achieved 97.5% Pass@1, outperforming the default ReAct baseline (82.5%) and rivaling proprietary frontier results.
- CVE-Bench: Reached 97.5% accuracy on zero-day tasks and 100% on one-day tasks.
- ExploitGym: Solved 157 tasks with GLM-5.3, significantly outperforming the Claude Code baseline (130 tasks).
- Cost Efficiency: This is the standout result. The GLM-5.2 configuration costs only $4.80 per case. As shown in the figures below, it achieves a much higher "yield" of successful exploits per dollar spent compared to closed-source models.

Limitations and Future Work
The authors noted a specific failure mode in userspace targets: agents would often capture a "flag" by finding a different vulnerability than the one intended (off-target captures). This suggests that in broad attack surfaces, directing the agent to a specific bug remains a challenge.
Future work will likely focus on the "co-evolution" of these multi-agent harnesses with even more specialized, domain-specific models to further push the boundaries of automated offensive security.
Summary updated