While AI coding agents have become remarkably proficient at writing software, the physical world presents a much steeper challenge. In the realm of integrated circuits, "working code" isn't enough—designs must obey the unforgiving laws of physics and geometry. To address this, a team from Nova-Silicon has introduced AnalogLayoutWorld, an open-source benchmark designed to push agentic AI into the complex domain of analog circuit layout.
TL;DR
AnalogLayoutWorld is a new open-source benchmark featuring 40 expert-reviewed tasks that challenge AI agents to generate analog circuit layouts from transistor-level netlists. The study reveals a significant "reality gap": while over 91% of agent attempts successfully produce a layout file, only 21.3% actually meet the rigorous physical and electrical criteria required for a functional chip.
What Problem Does This Paper Address?
Analog circuit layout is a "long-horizon" engineering task. Unlike digital design, which is highly automated, analog layout remains a boutique craft requiring deep domain knowledge and geometric reasoning. An engineer must place transistors and route wires while managing parasitic effects (unwanted resistance and capacitance) that can ruin a circuit's performance.
Current AI benchmarks mostly focus on software development or simple tool-use. They lack the "physicality" of hardware engineering, where a design must pass Design Rule Checks (DRC) for manufacturability and Layout-Versus-Schematic (LVS) checks for logical correctness. This paper introduces a standardized environment to evaluate if AI agents can navigate these constraints through iterative design and verification.
Methods
The authors developed a sandboxed, Docker-based environment pre-loaded with an open-source Electronic Design Automation (EDA) stack.
- The Workflow: Agents are given a fixed transistor netlist (the "blueprint") and specifications. They must write scripts to construct the layout using tools like
gdsfactoryorKLayout. - Iterative Feedback: Crucially, agents can run their own verification tools—DRC, LVS, and Parasitic Extraction (PEX)—to get feedback and refine their designs within a three-hour window.
- Independent Evaluation: Once the agent finishes, the benchmark performs an independent "blind" grading. This includes post-layout simulations using
ngspiceto ensure the circuit still meets electrical specs (like gain or bandwidth) after being turned into a physical layout.
Datasets
The benchmark consists of 40 expert-reviewed tasks based on the open-source SKY130 process technology. The tasks cover a diverse range of analog building blocks, including:
- Amplifiers (OTAs, Miller-compensated stages)
- Voltage references and power management circuits (LDOs)
- Sampling circuits and oscillators
Each task includes a verified electrical baseline, ensuring that if an agent fails, it is due to the layout process, not a faulty initial circuit design.
Key Insights and Ideas
- Artifact Generation $\neq$ Task Completion: The high rate of GDS (layout file) production (91.1%) vs. the low acceptance rate (21.3%) highlights that agents are good at "going through the motions" but struggle with the precision required for engineering closure.
- The Importance of Iteration: The study found that 90.3% of successful DRC passes and 66.8% of LVS passes occurred after the first verification call. This proves that the ability to interpret error logs and self-correct is the defining trait of a successful agent.
- Test-Time Scaling: GPT-series models (like GPT-6 Astra) showed a unique ability to scale their effort. As circuits got larger and more complex, these models increased their tool calls and token usage, maintaining higher quality than open-weight models which often "saturated" their budgets on simpler tasks.
- Engineering Quality Matters: Beyond just "passing," the benchmark measures "compactness" and "engineering quality" (e.g., matching transistors for symmetry). A layout might work electrically but be too large or poorly routed to be practical in a real product.
Key Results
The researchers evaluated 13 different model-harness configurations.
- Top Performer: Novara, a specialized internal agent system, achieved the highest aggregate score (0.8154), excelling particularly in LVS and engineering quality.
- General Purpose Agents: GPT-6 Astra followed closely (0.7483), leading in DRC pass rates and post-layout simulation.
- The Gap: Many open-source models struggled significantly, with some (like Kimi-K2.6) failing to produce any layouts that met all acceptance criteria.
Limitations and Future Work
The authors acknowledge several limitations:
- Fixed Netlists: The benchmark currently doesn't ask agents to size transistors or choose topologies—only to lay them out.
- Single PDK: It is currently limited to the SKY130 process. Future versions aim to include more advanced nodes like SG13G2.
- Verification Boundary: While rigorous, the benchmark uses rule-based checks that are proxies for real silicon performance. It does not guarantee that a passing design would work perfectly after actual fabrication (tape-out).
Future work will focus on expanding the task suite to include larger hierarchical circuits and refining the "engineering quality" metrics through further community feedback.
Summary updated