Graph Is the Verifier: Agentic Reinforcement Learning for Interprocedural Vulnerability Detection

Yikun Li, Ting Zhang, Jiakun Liu, Jinfeng Jiang, Yuheng Yieh, Yixin Yang, Wen Bin Leow, Yide Yin, Yintong Huo, Eng Lieh Ouh, Lwin Khin Shar, David LoarXiv 2026 · arXiv

TL;DR

VulAgentRL is an agentic reinforcement learning (RL) framework that detects complex, multi-function software vulnerabilities by navigating a Code Property Graph (CPG). By utilizing the CPG as both an interactive tool at inference and an exact evidence verifier during training, it prevents the common RL pitfall of "guessing" and significantly outperforms frontier LLMs.

What Problem Does This Paper Address?

Most learning-based vulnerability detectors evaluate functions in isolation. However, real-world security flaws rarely stay confined to a single block of code. The authors find that 71.7% of vulnerable functions require interprocedural evidence—context from callers, callees, or dataflow paths outside the function body—to be correctly diagnosed.

While large language models (LLMs) acting as autonomous agents could theoretically navigate codebases to find this evidence, training them via reinforcement learning is notoriously difficult. If you only reward the agent based on whether its final verdict ("vulnerable" or "safe") is correct, the model suffers from evidence-free degeneration. Instead of actually investigating, the agent learns to exploit class priors and simply guess the answer.

Methods

To solve this, the authors introduce VulAgentRL, which utilizes a Code Property Graph (CPG) in a dual role: as the agent's navigation tool and as the training environment's automated verifier.

Figure 1: VulAgentRL investigating a format-string vulner- ability in mpv. The agent queries the CPG to rule out a candidate guard, establishes that the format string originates from a user-supplied URL, and cites the graph nodes that justify its verdict.

The training pipeline consists of two main stages:

  1. Reasoning-and-Tool-Use SFT: The model (initialized from Qwen2.5-Coder-7B-Instruct) is first trained via Supervised Fine-Tuning on high-quality, multi-turn investigation trajectories distilled from a frontier teacher model (Claude Opus).
  2. GRPO RL with a CPG Verifier: The SFT model is optimized using Group Relative Policy Optimization (GRPO). Crucially, because every node in the CPG has a persistent integer identifier, the reward function can exactly match the agent's cited evidence nodes against the ground truth.

The composite reward function is strictly rule-based and balances three objectives: Verdict Agreement (60%), Evidence Grounding via $F_1$ score of node IDs (20%), and Common Weakness Enumeration (CWE) class agreement (20%).

Figure 3: Overview of the VulAgentRL training pipeline. (1) Stage 1: Reasoning-and-Tool-Use SFT. A teacher LLM in- vestigates each function by querying the CPG and produces multi-turn trajectories, which are filtered via rejection sam- pling to form a high-quality corpus; the student model is then fine-tuned to learn the query-then-report format. (2) Stage 2: GRPO RL with CPG Verifier. The SFT model is refined with reinforcement learning: for each function the policy generates parallel multi-turn rollouts, which are scored by a composite reward over verdict correctness, evidence ground- ing, and CWE class, with the cited evidence nodes verified against the same CPG the policy queried.

Datasets

  • PrimeVul: The primary corpus used for training and evaluation, containing 1,122 unique functions across 275 repositories. The authors apply a strict repository-level 8:1:1 split and per-commit deduplication to completely eliminate data leakage.
  • TitanVulOOD: A newly curated, clean out-of-distribution (OOD) test set containing 648 C/C++ functions across 217 repositories entirely disjoint from PrimeVul to evaluate cross-project generalization.

Key Insights and Ideas

  • The CPG as an Exact Verifier: Evaluating an agent's reasoning using an "LLM-as-a-judge" or fuzzy text matching is brittle and unreliable. By leveraging the persistent integer IDs of a CPG, evidence verification becomes a deterministic set-comparison problem.
  • SFT Warm-Starts are Structurally Mandatory: The authors demonstrate that RL cannot bootstrap tool-use behavior from a cold start. If a base model starts with zero tool calls, the group-relative advantage in GRPO provides no gradient signal to steer the policy toward querying. SFT initialization is required to bridge this gap.
Figure 5: Training curves of the GRPO stage: SFT-initialized vs. 0-shot cold start.
  • RL Breeds Efficiency: Interestingly, reinforcement learning didn't just make the model more accurate; it made it more efficient. The RL-tuned model dropped its average number of tool calls per rollout from 7.6 to 4.3, learning to conduct shorter, better-targeted investigations.

Key Results

VulAgentRL achieves state-of-the-art performance, outperforming both massive frontier models and specialized agent baselines on the strict Pair-Wise-Correct (P-C) metric (which requires classifying both the vulnerable and patched versions of a function correctly).

Table 1: Comparison of VulAgentRL with state-of-the-art baselines and its own training stages on the held-out test set.
  • Performance Leap: On the PrimeVul test set, VulAgentRL achieves a P-C score of 0.378, outperforming Claude Opus 4.7 (0.273) and the previous state-of-the-art agentic detector, JitVul (0.051).
  • OOD Generalization: On the unseen TitanVulOOD dataset, VulAgentRL maintains its edge with a P-C score of 0.302, proving the graph-grounded training transfers well to entirely new codebases.
  • Class Imbalance Robustness: In realistic scenarios where benign code heavily outnumbers vulnerable code (up to a 1:16 ratio), VulAgentRL consistently maintains a higher F1-score than its SFT and 0-shot counterparts.

Limitations and Future Work

While the underlying CPG substrate is inherently language-agnostic, the authors limited their current empirical evaluation strictly to C/C++ codebases. Additionally, VulAgentRL is currently optimized purely for vulnerability detection. The authors note that extending this graph-verified agentic framework to automated vulnerability repair (patching) is a compelling next step.

Summary updated

Abstract

Real-world vulnerabilities often span multiple functions, yet most learning-based detectors classify each function in isolation: on a sample of real CVEs, we find that 71.7% of vulnerable functions require evidence from outside the function to be classified correctly. Agentic reinforcement learning (RL) could close this gap by enabling a model to gather that evidence itself, but it lacks a reliable reward, since a reward defined on the final verdict alone can be obtained without performing any investigation. We propose VulAgentRL, an agentic RL framework for interprocedural vulnerability detection built on a Code Property Graph (CPG). The CPG serves two roles: at inference time the policy queries it for callers, callees, dataflow, and other queries, and at training time the same graph verifies the evidence the policy cites. Because every CPG node carries a persistent integer identifier, this verification is an exact comparison rather than a textual match, so the reward credits verdicts that are supported by evidence. We further initialize the policy by distilling teacher investigations, and show that this warm start is necessary, since RL cannot acquire tool-use behavior it never samples. Under a repository-level split that prevents leakage, VulAgentRL outperforms state-of-the-art baselines, including frontier models, on the strict pair-wise-correct metric while issuing fewer tool calls, and its advantage persists on an out-of-distribution corpus and under class imbalance.
BibTeX
@misc{li2026graph,
  author = {Yikun Li and Ting Zhang and Jiakun Liu and Jinfeng Jiang and Yuheng Yieh and Yixin Yang and Wen Bin Leow and Yide Yin and Yintong Huo and Eng Lieh Ouh and Lwin Khin Shar and David Lo},
  title = {{Graph Is the Verifier: Agentic Reinforcement Learning for Interprocedural Vulnerability Detection}},
  year = {2026},
  eprint = {2607.26656},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2607.26656},
}
Figure