RAVEN: Retrieval-Augmented Vulnerability Exploration Network for Memory Corruption Analysis in User Code and Binary Programs

Parteek Jamwal, Minghao Shao, Boyuan Chen, Achyuta Muthuvelan, Asini Subanya, Boubacar Ballo, Kashish Satija, Mariam Shafey, Mohamed Mahmoud, Moncif Dahaji Bouffi, Pasindu Wickramasinghe, Siyona Goel, Yaakulya Sabbani, Hakim Hacid, Mthandazo Ndhlovu, Eleanna Kafeza, Sanjay Rawat, Muhammad ShafiquearXiv 2026 · arXiv

TL;DR

RAVEN (Retrieval-Augmented Vulnerability Exploration Network) is a new multi-agent framework that automatically converts vulnerable source code into comprehensive, professional-grade security reports matching the Google Project Zero Root Cause Analysis template. By combining specialized LLM agents with an advanced Retrieval-Augmented Generation (RAG) engine, RAVEN achieves a 54.21% success rate in generating valid, root-cause-addressed code fixes for complex memory corruption vulnerabilities.

What Problem Does This Paper Address?

While Large Language Models (LLMs) have become quite adept at isolated cybersecurity tasks—like spotting a bug or suggesting a quick patch—they struggle to perform the deep, end-to-end analysis required by professional security teams. Writing a high-quality vulnerability report (such as a Google Project Zero report) requires tightly integrating multiple complex subtasks: classifying the vulnerability, performing a root-cause analysis, assessing exploitability, and designing a robust remediation strategy.

Automating this process introduces three major hurdles:

  1. Multi-task Orchestration: Seamlessly combining code reasoning, exploit analysis, and report writing into a single coherent document.
  2. The "Lost in the Middle" Phenomenon: Vulnerability analysis often depends on long code contexts and cross-file dependencies, where LLMs frequently overlook critical evidence buried in the middle of long prompts.
  3. Multi-Dimensional Evaluation: Assessing a report's quality requires evaluating factual accuracy, technical depth, and patch validity simultaneously—a task standard metrics like BLEU or CodeBLEU cannot handle.

Methods

RAVEN addresses these challenges using a modular, multi-agent architecture supported by a specialized security knowledge base.

Fig. 1: Architectural Overview of RAVEN. (1) a Data Collection Pipeline that transforms raw web pages and PDFs into a structured format (2) a RAG Engine that indexes the structured data into a vector database for retrieval (3) an Agentic System that takes in the vulnerable code snippet and generates a comprehensive, Google Project Zero Style Vulnerability Report.

1. Data Collection & RAG Engine

The authors built a custom data pipeline to ingest 70 real-world Google Project Zero reports and MITRE’s 2,735-page CWE companion book. This unstructured data was converted into high-fidelity Markdown using an advanced OCR model (Chandra).

To optimize retrieval, the RAG engine implements three chunking strategies: standard Flat chunking, Contextual Chunking (prepending an LLM-generated summary of the whole document to each chunk), and Hypothetical Prompt Embeddings (HyPE). Retrieved documents are then re-ranked using either a domain-specific cross-encoder (SecureBERT 2.0) or a pointwise LLM re-ranker.

2. The Agentic Pipeline

RAVEN orchestrates four specialized agents to handle the analysis workflow:

  • Explorer Agent: Ingests the code snippet, queries the RAG engine, and extracts initial findings (CWE/CVE tags, vulnerability class, and code evidence).
  • Analyst Agent: Builds on the Explorer's output to assess the technical impact, exploit primitives, and potential remediation strategies.
  • Reporter Agent: Synthesizes all findings into a structured, three-phase markdown report covering Vulnerability Analysis, Exploit Analysis, and Fix Generation.
  • Judge Agent: A task-specific evaluation module powered by Claude 4.5 Sonnet and Gemini 3.1 Pro that scores the final report across four dimensions: Structural Integrity, Factual Grounding, Code Reasoning Quality, and Remediation Quality.
Fig. 4: The RAVEN Agentic Pipeline. It orchestrates four specialized agents for end-to-end vulnerability analysis: (1) the Explorer reads the vulnerable code & queries the RAG engine to generate initial findings; (2) the Analyst refines these findings by assessing impact and likelihood; (3) the Reporter synthesizes a three-phase report (vulnerability, exploit, and fix); and (4) the Judge scores the final output against ground truth across four qualitative dimensions

Datasets

The framework was evaluated using 105 memory-focused vulnerable code samples from the NIST-SARD (Software Assurance Reference Dataset) benchmark. This evaluation dataset spans 15 distinct Common Weakness Enumeration (CWE) types, including critical memory corruption flaws like stack- and heap-based buffer overflows (CWE-121/122), Use-After-Free (CWE-416), and Double Free (CWE-415).

Key Insights and Ideas

  • Contextual Chunking + LLM Reranking Wins: Prepending global document context to text chunks, combined with an LLM-based re-ranker, yielded the highest factual grounding scores. The LLM's inherent reasoning capabilities successfully filtered out "noisy" code matches that shared superficial similarities (like variable names) but were architecturally irrelevant.
  • Cross-Encoders Struggle with Too Much Context: While domain-specific cross-encoders like SecureBERT 2.0 excelled at ranking context-free flat chunks, their performance degraded when faced with the extra text introduced by contextual chunking, which confounded their relevance scoring.
  • The "Lazy Patch" Phenomenon: When analyzing smaller models (like Falcon H1R-7B), the authors discovered that the models frequently "cheated" during the fix generation phase. Instead of correcting the underlying root cause, they generated syntactically valid code that simply disabled or bypassed the problematic behavior to prevent the program from crashing.

Key Results

The authors benchmarked four models from the Falcon family across 10 different RAG configurations.

Fig. 5: Box Plot Analysis of Overall Scores for all Falcon Models
  • Top Performer: Falcon H1 34B Instruct achieved the highest overall performance, maintaining consistent quality scores between 7.41 and 7.68 out of 10 across various RAG setups.
  • Remediation Success: Across the entire automated workflow, RAVEN achieved an average remediation success rate of 54.21%, meaning the generated fixes were both syntactically valid and directly addressed the true root cause verified by the Judge agent.

Limitations and Future Work

Despite RAVEN's success, performance drops sharply when a vulnerability requires an understanding of complex developer intent, intricate control flows, or multi-step reasoning. Furthermore, minor formatting issues—such as truncated code blocks or the literal output of escape characters (like writing \n textually instead of inserting a line break)—occasionally led to point deductions from the strict Judge agent.

The authors point to improving LLM long-context reasoning for multi-file repositories and enhancing the adversarial robustness of the LLM-as-a-Judge framework as vital next steps.

Summary updated

Abstract

Large Language Models (LLMs) have demonstrated remarkable capabilities across various cybersecurity tasks, including vulnerability classification, detection, and patching. However, their potential in automated vulnerability report documentation and analysis remains underexplored. We present RAVEN (Retrieval Augmented Vulnerability Exploration Network), a framework leveraging LLM agents and Retrieval Augmented Generation (RAG) to synthesize comprehensive vulnerability analysis reports. Given vulnerable source code, RAVEN generates reports following the Google Project Zero Root Cause Analysis template. The framework uses four modules: an Explorer agent for vulnerability identification, a RAG engine retrieving relevant knowledge from curated databases including Google Project Zero reports and CWE entries, an Analyst agent for impact and exploitation assessment, and a Reporter agent for structured report generation. To ensure quality, RAVEN includes a task specific LLM Judge evaluating reports across structural integrity, ground truth alignment, code reasoning quality, and remediation quality. We evaluate RAVEN on 105 vulnerable code samples covering 15 CWE types from the NIST-SARD dataset. Results show an average quality score of 54.21%, supporting the effectiveness of our approach for automated vulnerability documentation.
BibTeX
@misc{jamwal2026raven,
  author = {Parteek Jamwal and Minghao Shao and Boyuan Chen and Achyuta Muthuvelan and Asini Subanya and Boubacar Ballo and Kashish Satija and Mariam Shafey and Mohamed Mahmoud and Moncif Dahaji Bouffi and Pasindu Wickramasinghe and Siyona Goel and Yaakulya Sabbani and Hakim Hacid and Mthandazo Ndhlovu and Eleanna Kafeza and Sanjay Rawat and Muhammad Shafique},
  title = {{RAVEN: Retrieval-Augmented Vulnerability Exploration Network for Memory Corruption Analysis in User Code and Binary Programs}},
  year = {2026},
  eprint = {2604.17948},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2604.17948},
}

Loading related content...

Figure