Abstract
Smart contracts are the backbone of the Web 3.0 ecosystem, but their "code is law" nature means a single bug can lead to catastrophic financial loss. While detection tools have improved, simply identifying vulnerabilities is insufficient for developers who require comprehensive management. This paper introduces LLM-BSCVM, a framework that doesn't just detect vulnerabilities—it analyzes, risks, repairs, and verifies them using a collaborative team of AI agents. By coordinating six specialized agents and leveraging Retrieval-Augmented Generation (RAG), LLM-BSCVM achieves over 91% detection accuracy while providing automated code repairs and detailed audit reports.
Introduction
The security of decentralized finance (DeFi) is currently a game of cat and mouse. Traditional tools, such as symbolic execution or fuzzing, often suffer from high false-positive rates or struggle with complex logic. While Large Language Models (LLMs) have shown promise in code analysis, most existing research treats them as simple "bug hunters." Vulnerability management is a multi-dimensional problem: finding a bug is only the first step. Developers also need to understand the root cause, assess the risk level, implement a repair, and verify the fix. LLM-BSCVM addresses this gap by covering the entire vulnerability lifecycle.
Methodology: The Decompose-Retrieve-Generate Workflow
The core of LLM-BSCVM is a three-stage process designed to maximize accuracy and minimize hallucinations.
1. Task Decomposition
The complex job of security auditing is broken down into six sub-tasks, each handled by a specialized "Intelligent Agent":
- Detector: Identifies potential vulnerabilities using a mix of static analysis and LLM inference.
- Advisor: Analyzes the root cause and suggests repair strategies.
- Assessor: Assigns a risk level (Critical to Low) based on potential impact.
- Fixer: Generates the actual repaired Solidity code.
- Verifier: Acts as an independent auditor to check if the fix is valid.
- Reporter: Compiles findings into a comprehensive, human-readable audit report.
2. Knowledge Retrieval (RAG)
To prevent hallucinations, agents utilize Retrieval-Augmented Generation (RAG) to query external databases, including a Smart Contract Corpus and a Vulnerability Knowledge Base containing 263 expert audit reports.
3. Result Generation
The agents combine retrieved context with fine-tuned reasoning to produce explainable results rather than binary outputs. This specialization mimics a real-world security firm, creating a check-and-balance mechanism that improves reliability.
Key Results and Insights
The framework was evaluated using datasets from TrustLLM and Dappscan. Key findings include:
- High Accuracy: The framework achieved a vulnerability detection accuracy and F1 score of over 91%, outperforming base models like CodeLlama by nearly 40 percentage points.
- Superior Precision: It reached a precision of 94.95%, significantly reducing the False Positive Rate (FPR) to 5.1%.
- Automated Repair: Approximately 21% of contracts were successfully repaired to a level that passed independent verification.
- The Importance of RAG: Ablation studies confirmed that RAG is the most critical component; without external knowledge retrieval, performance dropped significantly.
Conclusion
LLM-BSCVM represents a significant step forward in automated smart contract security. By moving beyond simple detection to a full management lifecycle, it provides developers with actionable insights and verified repairs. Future work will focus on integrating formal verification for mathematical proofs of security and optimizing human-in-the-loop interactions.
Summary updated