SAEL: Leveraging Large Language Models with Adaptive Mixture-of-Experts for Smart Contract Vulnerability Detection

Lei Yu, Shiqi Cheng, Zhirong Huang, Jingyuan Zhang, Chenjie Shen, Junyi Lu, Li Yang, Fengjun Zhang, Jiajia Ma

TL;DR

SAEL is a novel framework that enhances smart contract vulnerability detection by combining the broad reasoning of general-purpose Large Language Models (LLMs) with the precision of specialized code models. By treating LLM-generated explanations as a core feature and using an Adaptive Mixture-of-Experts (MoE) architecture, SAEL achieves state-of-the-art results across major vulnerability types like reentrancy and delegatecall.

What Problem Does This Paper Address?

Smart contracts are the backbone of decentralized finance, but their "code is law" nature means a single bug can lead to irreversible multi-million dollar thefts (like the infamous DAO attack).

Current detection methods fall into three camps, each with a fatal flaw:

  1. Static Analysis (e.g., Slither): Fast but rigid. They often flag "false positives" because they don't understand high-level context (like an onlyOwner modifier that prevents an attack).
  2. Specialized Pre-trained Models: Great at spotting known patterns in specific datasets but struggle to generalize to new, complex code structures.
  3. General LLMs (e.g., GPT-4, Qwen): Excellent at "understanding" code logic and explaining why something might be a bug, but they often lack the surgical precision of models trained specifically for vulnerability detection.

The authors identify a missed opportunity: LLM explanations are often more accurate than their final "yes/no" predictions. Even if an LLM incorrectly labels a contract as vulnerable, its step-by-step analysis often contains the "ground truth" logic needed to make a correct final call.

Methods

The authors propose SAEL, a framework that integrates three distinct streams of information:

  1. Prompt Engineering with "Mimic-in-the-Background": They use Qwen1.5-72B-Chat with tailored prompts and Chain-of-Thought reasoning. To ensure reliability, they generate five responses in the background and select the most frequent one to mitigate LLM "hallucinations."
  2. Feature Extraction:
    • Raw Code: Processed via a prompt-tuned CodeT5 model.
  3. LLM Explanations: The natural language reasoning from the LLM is processed via a prompt-tuned T5 model.
  4. LLM Prediction: The binary (0 or 1) output from the LLM.
  5. Adaptive Mixture-of-Experts (MoE): This is the "brain" of SAEL. Instead of just averaging the results, a Gating Network dynamically weights these features for every specific contract. It uses a TopK mechanism to pick the most relevant features and Multi-Head Self-Attention to understand how the code and the explanations relate to each other.
  6. Optimized Loss Function: The model is trained to balance the independent performance of each feature with the overall weighted prediction, ensuring no single "expert" dominates incorrectly.

Datasets

The researchers evaluated SAEL on over 200,000 real-world smart contracts across four major vulnerability categories:

  • Reentrancy: Evaluated using the SmartBugs Wild Dataset.
  • Timestamp Dependency: Evaluated using the ESC (Ethereum Smart Contracts) Dataset.
  • Integer Overflow/Underflow & Delegatecall: Evaluated using a mix of two large-scale public datasets (Zhuang et al. and Liu et al.).

Key Insights and Ideas

  • Explanations as Data: The most striking insight is that natural language explanations are a powerful feature for machine learning. By "reading" the LLM's reasoning, the downstream MoE can catch nuances that raw code analysis misses.
  • Contextual Awareness: Unlike static tools, SAEL can recognize that a potentially dangerous function is actually safe because it's protected by specific access control modifiers (like onlyOwner).
  • Dynamic Weighting: The MoE architecture acknowledges that for some contracts, the raw code is the best indicator, while for others, the LLM's high-level reasoning is more critical. The model learns when to trust which expert.

Key Results

SAEL consistently outperformed 12 state-of-the-art baselines (including Slither, Mythril, and specialized models like Peculiar).

  • F1-Score Improvements: SAEL showed significant gains over the best-performing baselines:
  • +2.33% for Reentrancy
    • +3.16% for Timestamp Dependency
    • +10.67% for Integer Overflow/Underflow
    • +13.32% for Delegatecall
  • Zero-Shot Capability: In tests on entirely unseen data, SAEL maintained high F1-scores (81%–92%), proving it isn't just memorizing patterns but actually "understanding" the vulnerabilities.

Limitations and Future Work

  • Data Dependency: The framework currently requires a substantial amount of labeled training data to fine-tune the MoE and T5 components.
  • Computational Overhead: Running a 72B parameter LLM (Qwen) to generate explanations is more resource-intensive than traditional static analysis.
  • Future Directions: The authors plan to explore unsupervised pre-training on massive unlabeled smart contract datasets to reduce the reliance on human-labeled examples. They also aim to further optimize the hyperparameter tuning process (e.g., using Tree-structured Parzen Estimators).

Summary updated

Abstract

With the increasing security issues in blockchain, smart contract vulnerability detection has become a research focus. Existing vulnerability detection methods have their limitations: 1) Static analysis methods struggle with complex scenarios. 2) Methods based on specialized pre-trained models perform well on specific datasets but have limited generalization capabilities. In contrast, general-purpose Large Language Models (LLMs) demonstrate impressive ability in adapting to new vulnerability patterns. However, they often underperform on specific vulnerability types compared to methods based on specialized pre-trained models. We also observe that explanations generated by generalpurpose LLMs can provide fine-grained code understanding information, contributing to improved detection performance. Inspired by these observations, we propose SAEL, a LLMbased framework for smart contract vulnerability detection. First, we design prompts targeting specific smart contract vulnerabilities to guide general-purpose LLMs in detecting vulnerabilities and providing explanations. The detection results generated by LLMs serve as prediction features. Then, we employ prompt-tuning on CodeT5 and T5 respectively to process contract code and explanations, enhancing model performance on specific tasks. To leverage the strengths of each component, we introduce Adaptive Mixture-of-Experts, a dynamic architecture for smart contract vulnerability detection. This mechanism dynamically adjusts feature weights through a Gating Network, which selects the most relevant features by applying TopK filtering and Softmax normalization, and a Multi-Head Self-Attention mechanism, which enhances cross-feature relationships by processing multiple attention heads in parallel. This design ensures that prediction results for LLMs, explanation features, and contract code features are effectively integrated through gradient optimization. The loss function focuses on the independent prediction performance of each feature and the overall performance of weighted predictions. Experimental results show that SAEL outperforms existing methods in detecting various vulnerabilities.
BibTeX
@inproceedings{yu2025sael,
  author = {Lei Yu and Shiqi Cheng and Zhirong Huang and Jingyuan Zhang and Chenjie Shen and Junyi Lu and Li Yang and Fengjun Zhang and Jiajia Ma},
  title = {{SAEL: Leveraging Large Language Models with Adaptive Mixture-of-Experts for Smart Contract Vulnerability Detection}},
  booktitle = {ICSME},
  year = {2025},
  url = {https://openreview.net/forum?id=rYTbRcABrP},
}

Loading related content...

Figure