SiliconGym: A Versatile Reinforcement Learning Environment for Agentic EDA

Hejia Zhang, Sheng Lu, Yitong Ma, Zhongming Yu, Zhijing Wu, Wentao Ni, Chia-Tung Ho, Jishen ZhaoarXiv 2026 · arXiv

TL;DR

SiliconGym is the first versatile reinforcement learning (RL) environment designed for "agentic" hardware engineering, enabling AI agents to interact with complex Electronic Design Automation (EDA) toolchains. By decoupling task specifications from tool execution and providing robust resource management, it allows researchers to train agents that can perform multi-round repository repairs and full-system optimizations across both open-source and commercial backends.

What Problem Does This Paper Address?

The field of hardware design is increasingly turning to Large Language Models (LLMs) and Reinforcement Learning (RL) to automate tasks like generating Register-Transfer Level (RTL) code and optimizing Power, Performance, and Area (PPA). However, existing RL environments for EDA are often "task-specific silos." They are typically hardcoded for a single tool (like a specific simulator) and a single task (like module generation), making it difficult to scale to real-world engineering workflows.

Real-world hardware engineering is "agentic": an engineer doesn't just write a file once; they inspect tool feedback, revise multiple files across a repository, and run parallel tests. Previous pipelines lacked the infrastructure to preserve state across these tool calls, suffered from "fragile" reward signals due to resource interference (like out-of-memory errors during parallel runs), and left researchers wondering if gains achieved with open-source tools would actually hold up in a commercial, industry-standard environment.

Comparison of task-specific harnesses vs. SiliconGym's unified environment

Methods

SiliconGym introduces a modular architecture that separates the candidate state (the code being worked on), the task specification (what success looks like), and budgeted execution (how the tools are run).

Key technical innovations include:

  • Versioned Workspaces (Records): Instead of single-shot submissions, SiliconGym uses a "Record" system. This allows agents to edit, self-test, and revise code over multiple rounds, with the environment tracking the lineage of changes and feedback.
  • Toolchain-Independent Semantics: The environment uses adapters to translate high-level tasks (e.g., "verify this RTL") into tool-specific commands for either open-source backends (Icarus, Yosys) or commercial ones (Cadence Xcelium, Genus).
  • Budgeted Parallel Execution: To prevent one bad RTL candidate from crashing a training server, SiliconGym enforces strict memory and time limits using container-level controls.
  • Streaming Dispatch: Rather than waiting for an entire batch of LLM answers to be generated before starting EDA checks, SiliconGym dispatches them as they arrive, significantly reducing idle time.
Fig. 2. SiliconGym architecture. Qualified task definitions feed direct submissions or agent-edited Records. Submitted candidates are sealed and validated before EDA tools perform simulation, synthesis, formal checks, or report analysis. Public feedback supports subsequent rounds; command and output excerpts illustrate the interaction.

Datasets

The authors evaluated SiliconGym across a massive catalog of 3,470 definitions, incorporating several existing benchmarks and new reference flows:

  • RTL Generation: VerilogEval, RTLLM, and CodeV.
  • PPA Optimization: ChipSeek.
  • Repository Repair: HWE-Bench (including complex designs like Ibex and Rocket Chip).
  • Verification & Coverage: VeriThoughts, CorrectBench, and CVDP.
  • Full-Hierarchy Flows: Large-scale designs including the Earlgrey (OpenTitan SoC) and OpenTPU accelerator.

Key Insights and Ideas

  • The "Sim-to-Real" Gap in EDA: A major finding is that functional correctness transfers almost perfectly between open-source and commercial simulators (less than a 0.2% difference). However, PPA gains are much more volatile. Commercial synthesizers are highly optimized and often "absorb" the clever RTL rewrites that open-source tools might reward, meaning a "gain" in an open-source flow might vanish in a commercial one.
  • Simulation is Not Enough: The authors found that 57.4% of candidates that appeared to reduce area in simple simulations actually failed more rigorous formal verification or initialization checks. This highlights the need for "acceptance-gated" rewards that SiliconGym provides.
  • Resource Isolation Matters: Without strict memory budgets, parallel RL rollouts can suffer from "noisy" rewards where a valid design is failed simply because a neighboring process hogged the system memory.
Validation showing that functional correctness scores are nearly identical across different simulators

Key Results

  • Efficiency: SiliconGym’s streaming dispatch finished a 256-answer RL batch 21.9% faster than the native ChipSeek harness by overlapping generation and tool execution.
  • Scalability: The environment successfully handled full-hierarchy SoC flows (Earlgrey) with over 200,000 cells, reaching global placement on both open-source and commercial chains.
  • Agentic Success: In repository repair tasks, agents were able to perform multi-file edits and self-tests to fix bugs in complex RISC-V cores like Ibex and CVA6.
  • Cross-Toolchain Transfer: Models trained on open-source tools achieved functional accuracy on commercial Xcelium (44.08%) nearly identical to models trained directly on Xcelium (44.24%).
Runtime comparison showing the speedup from streaming dispatch

Limitations and Future Work

While SiliconGym bridges the gap between open-source and commercial tools, the authors note that commercial PPA gains remain difficult to achieve through module-level rewrites alone. The commercial tools are so efficient at logic optimization that they often reach the same "floor" regardless of minor RTL variations.

Future work may involve extending these agentic capabilities to higher-level architectural decisions or floorplanning, where the impact on PPA is more pronounced and less likely to be "optimized away" by the synthesizer. Additionally, while the environment supports commercial tools, the researchers acknowledge that access to these licenses remains a bottleneck for the broader academic community, making the proven transferability of open-source training even more critical.

Summary updated

Abstract

Reinforcement learning (RL) with electronic design automation (EDA) feedback is advancing large language models for hardware design and verification. Extending these gains to hardware-engineering agents requires environments that span tasks, preserve state across tool calls, and support large hierarchies alongside parallel rollouts. We present SiliconGym, the first versatile, cross-task and cross-toolchain RL environment for hardware design and verification. By decoupling versioned workspaces, budgeted execution, and acceptance-gated scoring, SiliconGym scales from module-level RTL, testbench, and PPA tasks to multi-round repository repair and full-hierarchy SoC flows. Streaming dispatch overlaps tool checks with generation, finishing a 256-answer RL batch 21.9% faster than native ChipSeek. Using interchangeable open-source and commercial backends, we show that open-source-trained functional correctness transfers cleanly to commercial simulation, while paired netlist evaluation reveals when open-source PPA gains survive commercial synthesis.
BibTeX
@misc{zhang2026silicongym,
  author = {Hejia Zhang and Sheng Lu and Yitong Ma and Zhongming Yu and Zhijing Wu and Wentao Ni and Chia-Tung Ho and Jishen Zhao},
  title = {{SiliconGym: A Versatile Reinforcement Learning Environment for Agentic EDA}},
  year = {2026},
  eprint = {2611.25658},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.25658},
}
Figure