Proof2Silicon: Prompt Repair for Verified Code and Hardware Generation via Reinforcement Learning

Manvi Jha, Jiaxin Wan, Deming ChenarXiv 2025 · DOI · arXiv

Introduction

Large Language Models (LLMs) have demonstrated remarkable capabilities in code generation. However, they often struggle with formal verification—the mathematical proof that code adheres strictly to its specifications. In domains like hardware design and safety-critical systems, "mostly correct" is insufficient; a single logical error can lead to catastrophic failures or prohibitively expensive silicon re-spins.

Researchers from the University of Illinois Urbana-Champaign have introduced Proof2Silicon, a framework that bridges this gap. By using a reinforcement learning (RL) agent to iteratively "repair" LLM prompts, the system generates formally verified code that is then automatically synthesized into hardware. This approach achieves up to a 72% success rate in end-to-end hardware generation without the need for expensive LLM fine-tuning.

The PREFACE Framework: RL-Driven Prompt Repair

The core innovation of Proof2Silicon is PREFACE, a prompt optimization loop that treats prompt engineering as a search problem. Instead of modifying the weights of a massive, "frozen" LLM (such as Gemini-2-Flash or GPT-4o), the framework employs a Small Language Model (SLM), like Qwen-1.5B, to act as an RL agent.

The PREFACE RL-driven prompt repair loop.

The SLM generates a "meta-prompt" for the frozen LLM. If the resulting code—written in the formal verification language Dafny—fails verification, the error logs from the SMT solver are fed back to the SLM. Using Proximal Policy Optimization (PPO), the SLM learns to perform surgical edits on the prompt to resolve specific logical errors over multiple iterations. This structured feedback provides a much richer signal than simple pass/fail metrics, allowing the agent to navigate the complex space of formal proofs effectively.

From Verified Code to Silicon

Once the Dafny code is verified, it enters the Hardware Synthesis Pipeline. This transition from high-level software verification to synthesizable hardware involves several critical steps:

  1. Compilation and Sanitization: The verified Dafny code is compiled into Python. An automated script then "sanitizes" the code, removing software-centric abstractions like complex class wrappers or dynamic memory allocations that do not map well to hardware.
  2. PyLog Integration: The sanitized code is decorated with PyLog attributes, which guide the High-Level Synthesis (HLS) process.
  3. Vivado HLS: Finally, the code is transformed into Register Transfer Level (RTL) using Vivado HLS, making it ready for FPGA or silicon implementation.

Evaluation and Key Results

The framework was evaluated using a 100-task subset of DafnyBench, covering algorithms such as sorting, arithmetic operations, and list manipulations.

  • Verification Success: The PREFACE RL loop improved Dafny verification success rates by up to 21% compared to static prompting. For instance, Gemini-2-Flash's success rate jumped from 33% to 55%.
  • Synthesis Efficiency: Proof2Silicon achieved a hardware synthesis success rate of 72.4% for programs that passed verification. The system is notably efficient, with an average synthesis time of 32–34 seconds per kernel and a peak memory footprint of approximately 700 MB.

Discussion and Future Directions

Proof2Silicon demonstrates that formal verification can be automated through intelligent prompting. By keeping the primary LLM frozen, the system remains model-agnostic and can easily integrate future, more capable models.

However, challenges remain. Approximately 30% of verified programs failed synthesis because they utilized constructs like recursion or dynamic "while" loops that current HLS tools cannot map to static hardware. Future work aims to incorporate hardware-specific metrics—such as area, power, and latency—directly into the RL reward function. This would enable the agent to optimize not just for logical correctness, but also for hardware efficiency, truly closing the loop between formal proofs and optimized silicon.

Summary updated

Abstract

Large Language Models (LLMs) have demonstrated impressive capabilities in automated code generation but frequently produce code that fails formal verification, an essential requirement for hardware and safety-critical domains. To overcome this fundamental limitation, we previously proposed PREFACE, a model-agnostic framework based on reinforcement learning (RL) that iteratively repairs the prompts provided to frozen LLMs, systematically steering them toward generating formally verifiable Dafny code without costly fine-tuning. This work presents Proof2Silicon, a novel end-to-end synthesis framework that embeds the previously proposed PREFACE flow to enable the generation of correctness-by-construction hardware directly from natural language specifications. Proof2Silicon operates by: (1) leveraging PREFACE's verifier-driven RL agent to optimize prompt generation iteratively, ensuring Dafny code correctness; (2) automatically translating verified Dafny programs into synthesizable high-level C using Dafny's Python backend and PyLog; and (3) employing Vivado HLS to produce RTL implementations. Evaluated rigorously on a challenging 100-task benchmark, PREFACE's RL-guided prompt optimization consistently improved Dafny verification success rates across diverse LLMs by up to 21%. Crucially, Proof2Silicon achieved an end-to-end hardware synthesis success rate of up to 72%, generating RTL designs through Vivado HLS synthesis flows. These results demonstrate a robust, scalable, and automated pipeline for LLM-driven, formally verified hardware synthesis, bridging natural-language specification and silicon realization.
BibTeX
@misc{jha2025proof,
  author = {Manvi Jha and Jiaxin Wan and Deming Chen},
  title = {{Proof2Silicon: Prompt Repair for Verified Code and Hardware Generation via Reinforcement Learning}},
  year = {2025},
  eprint = {2509.06239},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2509.06239},
}
Figure