Abstract
Automated hardware debugging in Verilog remains a significant challenge due to "bug signal dilution" in large modules. We present ARSP, a two-stage system that utilizes semantic partitioning to decompose complex Verilog modules into manageable fragments. By focusing LLM attention on localized logic, ARSP achieves a 77.92% success rate on industrial-scale designs, significantly outperforming general-purpose models and existing state-of-the-art tools.
Introduction
In the high-stakes domain of chip design (ASIC/FPGA), a single undetected bug can lead to millions of dollars in losses and months of schedule delays. While Large Language Models (LLMs) have revolutionized software debugging, they often struggle with the unique complexities of Verilog. A primary obstacle is bug signal dilution: in a large hardware module, the buggy code might represent less than 1% of the total tokens. When an LLM processes the entire file, its attention is diluted by hundreds of lines of irrelevant but semantically dense code, making it difficult to pinpoint and fix errors.
The ARSP Methodology
To address these challenges, we introduce ARSP (Automated Repair via Semantic Partitioning), a "divide and conquer" strategy designed to enhance the precision of LLM-based hardware debugging. The system operates in two distinct stages:
Step 1: Semantic Partitioning
A specialized Partition LLM, fine-tuned from DeepSeek-Coder-6.7B, analyzes the buggy Verilog code alongside its design specification. It splits the module into "semantically tight fragments"—small, cohesive blocks that perform a single micro-function, such as a reset logic block, an FSM state transition, or a specific counter update.
Step 2: Fragment Repair
A second Repair LLM examines each fragment individually. While it is provided with the entire module for global context, it is strictly instructed to only output the corrected version of its assigned fragment. This allows the model to "zoom in" on potential bug sites without losing the big picture.
Data Synthesis and Training
High-quality Verilog bug-fix pairs are notoriously rare. To overcome this data scarcity, we developed a Synthetic Data Framework. Using a pipeline involving Claude-3.7 and GPT-4, we generated over 24,000 training examples. We injected five common categories of hardware bugs into clean code: operator errors, numerical value errors, keyword errors, variable name errors, and edge/clock errors. This dataset was used to teach the models how to identify and fix localized errors within a broader context.
Evaluation and Results
We evaluated ARSP using a testset of 132 cases derived from the RTLLM and GenBen benchmarks. These cases average 219 lines of code, representing a significant increase in complexity compared to previous research. ARSP achieved a 77.92% pass@1 success rate, outperforming the previous state-of-the-art (MEIC) by over 35% and surpassing raw Claude-3.7 performance by nearly 38%.
Key Insights
- Locality of Bugs: A manual study of 145 real-world bugs revealed that 97.93% of them reside entirely within a single semantic fragment, validating the partitioning approach.
- Scalability: For modules exceeding 300 lines, semantic partitioning provided a 41.9% improvement in success rate. While baseline models like Claude-3.7 see performance drop significantly as code length increases, ARSP maintained a robust 65% success rate.
- Specialization: Off-the-shelf models performed worse when given fragments without specialized training, suggesting that ARSP’s fine-tuning is essential for localized repair within a global context.
Conclusion
ARSP demonstrates that "focus" is more critical than "memory" for large-scale hardware debugging. By aligning the debugging process with the inherent semantic structure of Verilog, ARSP provides a scalable and highly effective solution for modern chip design workflows. Future work will focus on integrating ARSP more deeply into industrial verification pipelines and expanding the diversity of supported bug types.
Summary updated