VeriLeaky: Navigating IP Protection vs Utility in Fine-Tuning for LLM-Driven Verilog Coding

Zeng Wang, Minghao Shao, Mohammed Nabeel, Prithwish Basu Roy, Likhitha Mankali, Jitendra Bhandari, Ramesh Karri, Ozgur Sinanoglu, Muhammed Shafique, Johann KnechtelarXiv 2025 · DOI · arXiv

Introduction

As Large Language Models (LLMs) revolutionize software engineering, the semiconductor industry is increasingly looking to leverage these tools for hardware description languages like Verilog. However, Verilog is a specialized domain with limited public high-quality data. To create "expert" models, companies often fine-tune base LLMs on their own proprietary Register-Transfer Level (RTL) designs. This practice introduces a significant security risk: LLMs tend to memorize training data, potentially allowing users to extract sensitive intellectual property (IP) through simple prompts.

The IP Leakage Problem

The VeriLeaky study investigates this critical trade-off between model utility and IP protection. When a model is fine-tuned on secret processor architectures or cryptographic accelerators, it may inadvertently regenerate those exact designs when prompted by an end-user.

The VeriLeaky framework demonstrates how proprietary IP used for fine-tuning can be leaked through LLM-generated outputs.

The researchers found that memorization is not merely a bug but a byproduct of the learning process. In many cases, the generated code was structurally and functionally identical to the proprietary training data, posing a multi-million dollar risk to hardware firms.

Methodology and Datasets

To quantify this risk, the authors developed a framework using LLaMA 3.1-8B. The study utilized a diverse dataset of 703 in-house IP modules, covering a wide range of hardware categories including cryptographic accelerators, processors, and bus architectures.

Distribution of the proprietary Verilog dataset used for fine-tuning, featuring 703 modules across 15 categories.

The assessment methodology involved three stages:

  1. Fine-Tuning: Training the model on a mix of public (RTLCoder) and proprietary data.
  2. Leakage Assessment: Measuring leakage using AST Similarity (syntax matching) and Formal Equivalence (functional matching using industry tools like Synopsys Formality).
  3. Defense Evaluation: Testing logic locking (specifically the ASSURE technique) to obfuscate training data and prevent the model from learning the "unlocked" secret design.

Key Insights: The Utility-Security Trade-off

The study revealed several critical insights into the behavior of fine-tuned hardware LLMs:

  • High Fidelity Leakage: Up to 46.52% of generated code showed significant structural similarity to proprietary data. Even benign prompts could trigger the output of memorized fragments.
  • Functional Equivalence: A major finding was that syntax-based checks are insufficient. Models often generate code that looks different but functions identically to the secret IP, necessitating formal verification for security audits.
  • The Cost of Defense: Applying logic locking to the training data did reduce leakage by approximately 13.84%. However, this obfuscation introduced "noise" that hindered the model's ability to learn general Verilog coding patterns, resulting in a utility drop of up to 10.81 percentage points in coding performance (pass@k).

Results and Conclusion

Fine-tuning on high-quality in-house IP remains highly effective for improving model performance, yielding an average gain of 5.43 percentage points over baseline models. However, the accompanying risk of IP leakage is substantial. While logic locking offers a partial defense, it creates a sharp performance penalty.

Future research must focus on more robust protection mechanisms, such as hardware-specific watermarking or privacy-preserving fine-tuning algorithms, to enable the safe deployment of LLM-driven hardware design tools without compromising the industry's most valuable assets.

Summary updated

Abstract

Large language models (LLMs) offer significant potential for coding, yet fine-tuning (FT) with curated data is essential for niche languages like Verilog. Using proprietary intellectual property (IP) for FT presents a serious risk, as FT data can be leaked through LLM inference. This leads to a critical dilemma for design houses: seeking to build externally accessible LLMs offering competitive Verilog coding, how can they leverage in-house IP to enhance FT utility while ensuring IP protection? For the first time in the literature, we study this dilemma. Using LLaMA 3.1-8B, we conduct in-house FT on a baseline Verilog dataset (RTLCoder) supplemented with our own in-house IP, which is validated through multiple tape-outs. To rigorously assess IP leakage, we quantify structural similarity (AST/Dolos) and functional equivalence (Synopsys Formality) between generated codes and our in-house IP. We show that our IP can indeed be leaked, confirming the threat. As defense, we evaluate logic locking of Verilog codes (ASSURE). This offers some level of protection, yet reduces the IP's utility for FT and degrades the LLM's performance. Our study shows the need for novel strategies that are both effective and minimally disruptive to FT, an essential effort for enabling design houses to fully utilize their proprietary IP toward LLM-driven Verilog coding.
BibTeX
@misc{wang2025verileaky,
  author = {Zeng Wang and Minghao Shao and Mohammed Nabeel and Prithwish Basu Roy and Likhitha Mankali and Jitendra Bhandari and Ramesh Karri and Ozgur Sinanoglu and Muhammed Shafique and Johann Knechtel},
  title = {{VeriLeaky: Navigating IP Protection vs Utility in Fine-Tuning for LLM-Driven Verilog Coding}},
  year = {2025},
  eprint = {2503.13116},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2503.13116},
}
Figure