LLM4Vuln: A Unified Evaluation Framework for Decoupling and Enhancing LLMs' Vulnerability Reasoning

Yuqiang Sun, Daoyuan Wu, Yue Xue, Han Liu, Wei Ma, Lyuye Zhang, Miaolei Shi, Yang Liu

Abstract

Large language models (LLMs) have demonstrated significant potential in vulnerability detection. However, it remains unclear whether their performance stems from inherent reasoning capabilities or external aids like knowledge retrieval and prompt engineering. This paper introduces LLM4Vuln, a unified evaluation framework that isolates a model's vulnerability reasoning from other capabilities. Through controlled experiments across 9,312 scenarios, we examine the impacts of knowledge enhancement, context supplementation, and prompt schemes. Our findings reveal a "reasoning model paradox" and demonstrate the framework's practical utility by identifying 14 zero-day vulnerabilities in active bug bounty programs.

Decoupling the "Brain" from the "Book"

As LLMs become central to automated security auditing, a critical question arises: Is the model truly reasoning through the code, or is it merely benefiting from a well-indexed database or a lucky prompt? Current research often conflates internal capability with external enhancements, making it difficult to benchmark models fairly.

LLM4Vuln addresses this by formalizing LLM-based detection as a function of five variables: target code, external knowledge, program context, prompt scheme, and instruction-following capability. This modular approach allows researchers to understand exactly why a specific configuration outperforms another.

The LLM4Vuln Framework

The framework consists of four pluggable components designed to test different facets of the detection process:

  1. Knowledge Retrieval: Uses vector databases to provide models with raw vulnerability reports or summarized "root cause" knowledge.
  2. Context Supplementation: Automatically provides related functions (callees) to help the model understand broader program logic.
  3. Prompt Schemes: Compares simple "Raw" prompts against "Chain-of-Thought" (CoT) instructions.
  4. Instruction Following: A standardized pipeline that uses a high-capability model (e.g., GPT-4) as an "annotator" to convert unstructured outputs into structured data for objective comparison.
The four modular components of the LLM4Vuln evaluation framework.

The UniVul Benchmark

To support this modular evaluation, the researchers developed UniVul, a benchmark spanning three distinct domains:

  • Solidity: Focuses on complex smart contract business logic.
  • Java: Targets traditional software weaknesses like injection.
  • C/C++: Focuses on memory errors and low-level vulnerabilities.

UniVul provides a curated knowledge base for each language, simulating a scenario where a model has access to a library of known vulnerabilities during the reasoning process.

Key Insights and Experimental Results

The "Reasoning Model" Paradox

One of the most surprising findings involves "deep reasoning" models like DeepSeek-R1 and QwQ-32B. Unlike traditional foundation models, these advanced models often perform worse when provided with external knowledge or extra code context. They appear to have internal models of vulnerability semantics that are robust enough that additional information acts as a distraction rather than an aid.

Language-Specific Dynamics

Knowledge enhancement is a significant performance booster for Solidity, nearly doubling F1 scores for foundation models. Because smart contract bugs are often logic-based, providing analogous cases via Retrieval-Augmented Generation (RAG) helps bridge the reasoning gap. Conversely, for Java and C/C++, where bugs often follow well-known CWE patterns, pre-training is usually sufficient.

Precision via Chain-of-Thought

Chain-of-Thought (CoT) prompting was found to be more effective at increasing precision than recall. By forcing the model to justify its logic step-by-step, CoT helps the model discard "hallucinated" vulnerabilities, thereby reducing false positives.

Performance metrics across various LLMs and experimental setups, illustrating the impact of different configurations on detection accuracy.

Real-World Impact: Zero-Day Discoveries

To validate the framework's practical utility, the authors applied LLM4Vuln to active bug bounty programs. The system successfully identified 14 zero-day vulnerabilities across four projects (Apebond, Glyph AMM, StakeStone, and Hajime), resulting in $3,576 in bounties. This demonstrates that decoupling and properly enhancing LLM reasoning can lead to tangible security improvements in production environments.

Future Directions

While LLM4Vuln provides a robust baseline, challenges remain in retrieval accuracy—currently, the system only finds truly relevant knowledge for 36% of Java samples. Future work will focus on moving beyond text similarity toward functional similarity matching and integrating LLMs with traditional static analysis tools to better handle massive codebases.

Summary updated

Abstract

Large language models (LLMs) have demonstrated significant potential in various tasks, including vulnerability detection. However, current efforts in this area are preliminary, lacking clarity on whether LLMs' vulnerability reasoning capabilities stem from the models themselves or external aids such as knowledge retrieval and tooling support. This paper aims to isolate LLMs' vulnerability reasoning from other capabilities, such as vulnerability knowledge adoption, context information retrieval, and structured output generation. We introduce LLM4Vuln, a unified evaluation framework that separates and assesses LLMs' vulnerability reasoning capabilities and examines improvements when combined with other enhancements. We conducted controlled experiments with 97 ground-truth vulnerabilities and 97 non-vulnerable cases in Solidity and Java, testing them in a total of 9,312 scenarios across four LLMs (GPT-4, GPT-3.5, Mixtral, and Llama 3). Our findings reveal the varying impacts of knowledge enhancement, context supplementation, prompt schemes, and models. Additionally, we identified 14 zero-day vulnerabilities in four pilot bug bounty programs, resulting in \$3,576 in bounties.
BibTeX
@misc{sun2024llm,
  author = {Yuqiang Sun and Daoyuan Wu and Yue Xue and Han Liu and Wei Ma and Lyuye Zhang and Miaolei Shi and Yang Liu},
  title = {{LLM4Vuln: A Unified Evaluation Framework for Decoupling and Enhancing LLMs' Vulnerability Reasoning}},
  year = {2024},
  eprint = {2401.16185},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2401.16185},
}
Figure