Introduction
As Large Language Models (LLMs) revolutionize software engineering, the semiconductor industry is increasingly looking to leverage these tools for hardware description languages like Verilog. However, Verilog is a specialized domain with limited public high-quality data. To create "expert" models, companies often fine-tune base LLMs on their own proprietary Register-Transfer Level (RTL) designs. This practice introduces a significant security risk: LLMs tend to memorize training data, potentially allowing users to extract sensitive intellectual property (IP) through simple prompts.
The IP Leakage Problem
The VeriLeaky study investigates this critical trade-off between model utility and IP protection. When a model is fine-tuned on secret processor architectures or cryptographic accelerators, it may inadvertently regenerate those exact designs when prompted by an end-user.
The researchers found that memorization is not merely a bug but a byproduct of the learning process. In many cases, the generated code was structurally and functionally identical to the proprietary training data, posing a multi-million dollar risk to hardware firms.
Methodology and Datasets
To quantify this risk, the authors developed a framework using LLaMA 3.1-8B. The study utilized a diverse dataset of 703 in-house IP modules, covering a wide range of hardware categories including cryptographic accelerators, processors, and bus architectures.
The assessment methodology involved three stages:
- Fine-Tuning: Training the model on a mix of public (RTLCoder) and proprietary data.
- Leakage Assessment: Measuring leakage using AST Similarity (syntax matching) and Formal Equivalence (functional matching using industry tools like Synopsys Formality).
- Defense Evaluation: Testing logic locking (specifically the ASSURE technique) to obfuscate training data and prevent the model from learning the "unlocked" secret design.
Key Insights: The Utility-Security Trade-off
The study revealed several critical insights into the behavior of fine-tuned hardware LLMs:
- High Fidelity Leakage: Up to 46.52% of generated code showed significant structural similarity to proprietary data. Even benign prompts could trigger the output of memorized fragments.
- Functional Equivalence: A major finding was that syntax-based checks are insufficient. Models often generate code that looks different but functions identically to the secret IP, necessitating formal verification for security audits.
- The Cost of Defense: Applying logic locking to the training data did reduce leakage by approximately 13.84%. However, this obfuscation introduced "noise" that hindered the model's ability to learn general Verilog coding patterns, resulting in a utility drop of up to 10.81 percentage points in coding performance (pass@k).
Results and Conclusion
Fine-tuning on high-quality in-house IP remains highly effective for improving model performance, yielding an average gain of 5.43 percentage points over baseline models. However, the accompanying risk of IP leakage is substantial. While logic locking offers a partial defense, it creates a sharp performance penalty.
Future research must focus on more robust protection mechanisms, such as hardware-specific watermarking or privacy-preserving fine-tuning algorithms, to enable the safe deployment of LLM-driven hardware design tools without compromising the industry's most valuable assets.
Summary updated