SmartCoder-R1: Towards Secure and Explainable Smart Contract Generation with Security-Aware Group Relative Policy Optimization

Lei Yu, Jingyuan Zhang, Xin Wang, Jiajia Ma, Li Yang, Fengjun ZhangarXiv 2025 · arXiv · OpenReview

TL;DR

This paper introduces SmartCoder-R1, a novel framework that generates secure and explainable smart contracts by combining specialized pre-training, chain-of-thought reasoning, and reinforcement learning. It significantly outperforms existing models in generating code that is not only functional and compilable but also demonstrably secure, addressing a critical gap in blockchain security.

What Problem Does This Paper Address?

Smart contracts are crucial for automating high-value transactions on blockchains, but their vulnerabilities can lead to catastrophic financial losses. Current Large Language Models (LLMs) used for smart contract generation suffer from two major drawbacks: they act as "black boxes" without providing transparent reasoning, and consequently, they often produce code riddled with critical security flaws. This lack of explainability makes it difficult for developers to audit and trust the generated code, while the security vulnerabilities pose a direct threat to the integrity of blockchain applications.

Methods

SmartCoder-R1 employs a three-stage training pipeline:

  1. Continual Pre-training (CPT): The base LLM (Qwen2.5-Coder-7B) is further trained on a large corpus of Solidity code to specialize its understanding of the language's syntax and structure.
  2. Long Chain-of-Thought Supervised Fine-Tuning (L-CoT SFT): This stage explicitly teaches the model to emulate human security analysis. It's trained on expert-validated samples that include a detailed, step-by-step reasoning process (using <think>...</think> tags) before generating the final code (<answer>...</answer>). This instills a structured reasoning capability.
  3. Security-Aware Group Relative Policy Optimization (S-GRPO): This reinforcement learning phase refines the generation policy. The model generates multiple candidate outputs for a given prompt, and these are evaluated using a weighted reward function that prioritizes compilation success, security compliance, and correct formatting. This process directly optimizes for secure and functional code.

Datasets

  • Continual Pre-training (CPT) Dataset: A large corpus of 286,397 unique Ethereum smart contracts, augmented with general code, mathematics, and text in English and Chinese, totaling 620.56M tokens.
  • L-CoT SFT Dataset: 7,998 meticulously annotated samples, each including a contract context, functional requirements, an explicit Chain-of-Thought reasoning trace, and a reference implementation. These are derived from 1,508 unique contracts and have been manually verified for correctness.
  • S-GRPO Dataset: 1,691 samples specifically constructed for the reinforcement learning stage. Each sample includes a contract context, requirement, reasoning chain, and a labeled implementation to provide step-by-step feedback.
  • Evaluation Dataset: 756 unique functions from 289 real-world contracts, sourced from Ethereum mainnet and EtherScan. This dataset is kept strictly separate from all training data to ensure rigorous evaluation.

Key Insights and Ideas

  • Reasoning is Key: The paper strongly advocates that for security-critical domains like smart contracts, explicit reasoning processes are not just helpful but essential. Models that can articulate why they generate code in a certain way are far more trustworthy and secure.
  • Three-Stage Synergy: The combination of CPT, L-CoT SFT, and S-GRPO creates a powerful synergy. CPT builds foundational knowledge, L-CoT SFT teaches structured reasoning, and S-GRPO fine-tunes the model for optimal security and functionality through targeted feedback.
  • S-GRPO's Impact: The reinforcement learning stage (S-GRPO) is shown to be the most critical for achieving high security scores, significantly reducing vulnerability rates and boosting overall correctness.
  • Human Evaluation Matters: The generated reasoning chains are not only evaluated quantitatively but also qualitatively by human experts, confirming their superior clarity and security focus compared to baselines.

Key Results

SmartCoder-R1 achieves state-of-the-art performance on a benchmark of 756 real-world smart contract functions:

  • FullRate (compilable, secure, and functional): 50.53%, a 45.79% relative improvement over the strongest baseline (DeepSeek-R1 at 34.66%).
  • VulRate (vulnerability rate): 8.60% (lower is better), significantly outperforming baselines.
  • ComPass (compilability): 87.70%.
  • SafeAval (compilable and secure): 80.16%.
  • FuncRate (compilable and functional): 53.84%.

Human evaluations of reasoning chains also show high-quality ratings:

  • Functionality: 82.7%
  • Security: 85.3%
  • Clarity: 90.7%

Limitations and Future Work

  • Reward Function Nuances: The authors acknowledge that the S-GRPO reward mechanism, while effective, might not fully capture all aspects of code quality, such as conciseness or stylistic elegance, potentially leading to overly verbose outputs.
  • Dataset Generalizability: The training and evaluation datasets, while comprehensive, represent a subset of the evolving smart contract landscape. Performance on highly novel or out-of-distribution contract types might vary.
  • Future Work: The paper suggests refining the reward function to address verbosity, enhancing training data with more complex security counterexamples, integrating advanced static analysis tools into the reward mechanism, and further refining the security reasoning framework to tackle specific high-level vulnerabilities.

Summary updated

Abstract

Smart contracts automate the management of high-value assets, where vulnerabilities can lead to catastrophic financial losses. This challenge is amplified in Large Language Models (LLMs) by two interconnected failures: they operate as unauditable "black boxes" lacking a transparent reasoning process, and consequently, generate code riddled with critical security vulnerabilities. To address both issues, we propose SmartCoder-R1 (based on Qwen2.5-Coder-7B), a novel framework for secure and explainable smart contract generation. It begins with Continual Pre-training (CPT) to specialize the model. We then apply Long Chain-of-Thought Supervised Fine-Tuning (L-CoT SFT) on 7,998 expert-validated reasoning-and-code samples to train the model to emulate human security analysis. Finally, to directly mitigate vulnerabilities, we employ Security-Aware Group Relative Policy Optimization (S-GRPO), a reinforcement learning phase that refines the generation policy by optimizing a weighted reward signal for compilation success, security compliance, and format correctness. Evaluated against 17 baselines on a benchmark of 756 real-world functions, SmartCoder-R1 establishes a new state of the art, achieving top performance across five key metrics: a ComPass of 87.70%, a VulRate of 8.60%, a SafeAval of 80.16%, a FuncRate of 53.84%, and a FullRate of 50.53%. This FullRate marks a 45.79% relative improvement over the strongest baseline, DeepSeek-R1. Crucially, its generated reasoning also excels in human evaluations, achieving high-quality ratings for Functionality (82.7%), Security (85.3%), and Clarity (90.7%).
BibTeX
@misc{yu2025smartcoder,
  author = {Lei Yu and Jingyuan Zhang and Xin Wang and Jiajia Ma and Li Yang and Fengjun Zhang},
  title = {{SmartCoder-R1: Towards Secure and Explainable Smart Contract Generation with Security-Aware Group Relative Policy Optimization}},
  year = {2025},
  eprint = {2509.09942},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2509.09942},
}
Figure