Razavi-Bench: Tracking AI Progress in Analog Circuit Reasoning

Zhishuai Zhang, Oscar Toomey, Taekwang JangarXiv 2026 · arXiv

TL;DR

Razavi-Bench tracks the meteoric rise of AI in analog circuit reasoning, showing that top-tier models improved from 28.8% to 98.3% accuracy over a 30-month period ending in late 2026. While "textbook" circuits are now effectively solved, the benchmark reveals that the true frontier lies in "look-alike" circuits that punish models for relying on visual pattern recognition rather than first-principles analysis.

What Problem Does This Paper Address?

While AI progress in digital design (like writing Verilog) has been well-documented, analog circuit reasoning remains a formidable challenge. Analog design requires more than just code generation; it demands the ability to interpret complex schematics, identify subtle physical mechanisms, and predict how small design changes ripple through a system.

Existing benchmarks often focus on digital code or simple numerical answers. The authors identify a gap in testing reasoning: can a model tell the difference between a standard textbook circuit and a "look-alike" that appears familiar but functions differently due to a swapped transistor type or an inverted sign convention? Razavi-Bench aims to track this specific capability across the rapidly evolving landscape of multimodal large language models (MLLMs).

Methods

The researchers developed Razavi-Bench, a benchmark consisting of 50 questions derived from two influential articles by analog expert Behzad Razavi ("Analog Design Experiments With AI").

Key methodological features include:

  • Expert-Led Ground Truth: Questions were originally written and graded by Razavi himself. The authors calibrated their reference answers and rubrics against his professional verdicts.
  • LLM-as-a-Judge: They utilized GPT-6 Luna (at maximum reasoning effort) to grade 73 different models against these rubrics.
  • Longitudinal Tracking: The study spans 73 models released between May 2024 and November 2026, providing a rare look at the trajectory of domain-specific AI performance.
  • Zero-Shot Evaluation: Models were given the schematic and question without system prompts, tool access, or web search, forcing them to rely on internal reasoning.

Datasets

The primary dataset is the newly introduced Razavi-Bench (50 questions). The paper situates this alongside several other circuit-related benchmarks:

  • Digital/Code focused: VerilogEval, RTLLM.
  • Design focused: AnalogCoder.
  • Perception/Analysis focused: CIRCUIT (510 questions), AMSbench (8,000 questions), MMCircuitEval (3,614 questions), and CircuChain (which uses sign-inversion variants).

Key Insights and Ideas

  • Pattern Recognition vs. True Analysis: The most striking finding is that many models "recognize" a circuit's shape before analyzing its components. For example, a model might see a stack of two transistors and immediately label it a "cascode," even if the bottom transistor is a PMOS with a grounded drain (making it a source follower).
  • The "Look-Alike" Trap: As shown in Figure 2, models perform significantly worse on circuits that resemble textbook examples but have critical differences. Top models like Gemini 4 Carbon have finally overcome this, checking individual device symbols consistently, whereas mid-tier models often revert to familiar names even after identifying the correct transistor types.
Figure 2: Comparison of textbook circuits vs. look-alikes. The red "look-alikes" show a massive drop in mean score for most models.
  • Rapid Commoditization of Reasoning: The cost to reach 90% accuracy on this benchmark plummeted 17-fold in just five months. High-level reasoning, once the exclusive domain of the most expensive frontier models, is quickly becoming affordable.
  • The Expensive "Last Mile": While reaching 90% is getting cheaper, the final few percentage points of accuracy remain extremely costly. The Pareto front in Figure 3 shows that the most capable models (Gemini 4 Carbon, Claude Opus 5.5) require significantly more tokens and higher API costs to eliminate their remaining errors.

Key Results

  • Performance Leap: The top score on the benchmark rose from 28.83% to 98.33% in 30 months.
  • State of the Art: As of late 2026, Gemini 4 Carbon and Claude Opus 5.5 are the leaders, answering nearly every question correctly and showing high consistency across trials.
  • Timeline of Progress: Figure 1 illustrates the "step-change" nature of these improvements, with significant gains occurring even before reference answers were publicly available, suggesting these improvements stem from general reasoning advances rather than data leakage.
Figure 1: Overall scores of 73 models by release date, showing the rapid ascent toward near-perfect scores.
  • Cost Efficiency: By November 2026, models like Hy4 achieved >90% scores at a fraction of the cost of the previous year's leaders (Table 2).

Limitations and Future Work

  • Scale: With only 50 questions, the benchmark is relatively small and may eventually suffer from saturation as models reach near-perfect scores.
  • Static Analysis: The questions are self-contained and "short-horizon." The authors note that real-world analog design is an iterative, "agentic" process involving simulation and revision, which this benchmark does not capture.
  • Single Judge: The reliance on a single LLM (GPT-6 Luna) for grading, while calibrated, remains a potential point of bias.
  • Future Direction: The authors suggest moving toward "agentic design benchmarks" (like Analog Design Bench) that test a model's ability to use tools and handle hours-long design tasks.

Summary updated

Abstract

We propose Razavi-Bench, a benchmark built from the 50 questions in Razavi's two Analog Design Experiments With AI articles and graded by GPT-6 Luna against reference answers and rubrics, and use it to track how fast AI models improve at analog circuit analysis. Across 73 multimodal large language models released between May 2024 and November 2026, the top score rose from 28.83% to 98.33% in 30 months, mostly before the reference answers were published. The leading models now answer almost every textbook question correctly; the hardest questions show a circuit that resembles a textbook one but works differently, or ask about behavior the obvious analysis misses. Within two and a half years, a set of questions that exposed the limits of AI models has become one they answer almost perfectly.
BibTeX
@misc{zhang2026razavi,
  author = {Zhishuai Zhang and Oscar Toomey and Taekwang Jang},
  title = {{Razavi-Bench: Tracking AI Progress in Analog Circuit Reasoning}},
  year = {2026},
  eprint = {2611.20935},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.20935},
}
Figure