AnalogLayoutWorld: An Open Benchmark for Agentic Analog Layout Design

NovaSilicon TeamarXiv 2026 · arXiv

While AI coding agents have become remarkably proficient at writing software, the physical world presents a much steeper challenge. In the realm of integrated circuits, "working code" isn't enough—designs must obey the unforgiving laws of physics and geometry. To address this, a team from Nova-Silicon has introduced AnalogLayoutWorld, an open-source benchmark designed to push agentic AI into the complex domain of analog circuit layout.

TL;DR

AnalogLayoutWorld is a new open-source benchmark featuring 40 expert-reviewed tasks that challenge AI agents to generate analog circuit layouts from transistor-level netlists. The study reveals a significant "reality gap": while over 91% of agent attempts successfully produce a layout file, only 21.3% actually meet the rigorous physical and electrical criteria required for a functional chip.

What Problem Does This Paper Address?

Analog circuit layout is a "long-horizon" engineering task. Unlike digital design, which is highly automated, analog layout remains a boutique craft requiring deep domain knowledge and geometric reasoning. An engineer must place transistors and route wires while managing parasitic effects (unwanted resistance and capacitance) that can ruin a circuit's performance.

Current AI benchmarks mostly focus on software development or simple tool-use. They lack the "physicality" of hardware engineering, where a design must pass Design Rule Checks (DRC) for manufacturability and Layout-Versus-Schematic (LVS) checks for logical correctness. This paper introduces a standardized environment to evaluate if AI agents can navigate these constraints through iterative design and verification.

The agentic analog layout design process involves moving from a netlist definition to iterative physical design and eventual validation.

Methods

The authors developed a sandboxed, Docker-based environment pre-loaded with an open-source Electronic Design Automation (EDA) stack.

  • The Workflow: Agents are given a fixed transistor netlist (the "blueprint") and specifications. They must write scripts to construct the layout using tools like gdsfactory or KLayout.
  • Iterative Feedback: Crucially, agents can run their own verification tools—DRC, LVS, and Parasitic Extraction (PEX)—to get feedback and refine their designs within a three-hour window.
  • Independent Evaluation: Once the agent finishes, the benchmark performs an independent "blind" grading. This includes post-layout simulations using ngspice to ensure the circuit still meets electrical specs (like gain or bandwidth) after being turned into a physical layout.
Figure 3: Overview of AnalogLayoutWorld. Agents receive fixed circuit inputs and interact with a preconfigured EDA workspace through the native tools from harnesses. After termination or time- out, the submitted GDS undergoes independent physical, electrical, and layout-quality assessment. The waveform is cropped from KiCad [KiCad Documentation Contributors, n.d.].

Datasets

The benchmark consists of 40 expert-reviewed tasks based on the open-source SKY130 process technology. The tasks cover a diverse range of analog building blocks, including:

  • Amplifiers (OTAs, Miller-compensated stages)
  • Voltage references and power management circuits (LDOs)
  • Sampling circuits and oscillators

Each task includes a verified electrical baseline, ensuring that if an agent fails, it is due to the layout process, not a faulty initial circuit design.

Key Insights and Ideas

  • Artifact Generation $\neq$ Task Completion: The high rate of GDS (layout file) production (91.1%) vs. the low acceptance rate (21.3%) highlights that agents are good at "going through the motions" but struggle with the precision required for engineering closure.
  • The Importance of Iteration: The study found that 90.3% of successful DRC passes and 66.8% of LVS passes occurred after the first verification call. This proves that the ability to interpret error logs and self-correct is the defining trait of a successful agent.
  • Test-Time Scaling: GPT-series models (like GPT-6 Astra) showed a unique ability to scale their effort. As circuits got larger and more complex, these models increased their tool calls and token usage, maintaining higher quality than open-weight models which often "saturated" their budgets on simpler tasks.
  • Engineering Quality Matters: Beyond just "passing," the benchmark measures "compactness" and "engineering quality" (e.g., matching transistors for symmetry). A layout might work electrically but be too large or poorly routed to be practical in a real product.

Key Results

The researchers evaluated 13 different model-harness configurations.

  • Top Performer: Novara, a specialized internal agent system, achieved the highest aggregate score (0.8154), excelling particularly in LVS and engineering quality.
  • General Purpose Agents: GPT-6 Astra followed closely (0.7483), leading in DRC pass rates and post-layout simulation.
  • The Gap: Many open-source models struggled significantly, with some (like Kimi-K2.6) failing to produce any layouts that met all acceptance criteria.
Figure 2: Performance comparison of some agent systems across multiple verification metrics. DRC, LVS, and Post-simulation report pass rates. Score is the mean total over all 40 tasks.

Limitations and Future Work

The authors acknowledge several limitations:

  • Fixed Netlists: The benchmark currently doesn't ask agents to size transistors or choose topologies—only to lay them out.
  • Single PDK: It is currently limited to the SKY130 process. Future versions aim to include more advanced nodes like SG13G2.
  • Verification Boundary: While rigorous, the benchmark uses rule-based checks that are proxies for real silicon performance. It does not guarantee that a passing design would work perfectly after actual fabrication (tape-out).

Future work will focus on expanding the task suite to include larger hierarchical circuits and refining the "engineering quality" metrics through further community feedback.

Summary updated

Abstract

Coding agents have demonstrated strong capabilities in software development, yet their ability to tackle long-horizon engineering tasks governed by physical constraints remains less understood. We study this question through analog circuit layout, which demands domain knowledge, geometric reasoning, and iterative interaction with design and verification tools. We introduce \textbf{AnalogLayoutWorld}, an open-source agent benchmark with 40 expert-reviewed tasks for end-to-end layout generation from fixed transistor-level netlists. It combines an integrated design environment with independent physical and post-layout electrical verification, as well as objective layout quality metrics co-designed with analog layout experts. Across 13 model--harness configurations, 91.1\% of attempts produce GDS layouts, but only 23.0\% satisfy the benchmark's acceptance criteria. We also evaluate Novara, an internally developed agent system, which achieves the highest reported aggregate score. Our analysis links performance differences to test-time scaling under fixed budgets, suggesting that reasoning efficiency and effective use of verification feedback help sustain design quality as task complexity grows.
BibTeX
@misc{team2026analoglayoutworld,
  author = {NovaSilicon Team},
  title = {{AnalogLayoutWorld: An Open Benchmark for Agentic Analog Layout Design}},
  year = {2026},
  eprint = {2611.16991},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.16991},
}
Figure