ADO-LLM: Analog Design Bayesian Optimization with In-Context Learning of Large Language Models

Yuxuan Yin, Yu Wang, Boxun Xu, Peng Li

Introduction

Analog circuit sizing—the determination of optimal transistor dimensions, resistance, and capacitance—is a critical yet labor-intensive phase of integrated circuit design. Designers must navigate a complex trade-off space involving gain, power consumption, area, and speed. While automated tools have emerged, they often fall into two categories: systematic but "physics-blind" mathematical optimizers, or intuitive but inconsistent human-like agents.

Traditional Bayesian Optimization (BO) utilizes Gaussian Process (GP) models to explore design spaces. However, BO treats the circuit as a black box, often evaluating physically nonsensical designs. Large Language Models (LLMs), trained on vast repositories of engineering knowledge, offer a form of "design intuition" but struggle with the precision and systematic search required for fine-tuning. This paper presents ADO-LLM, a framework that harmonizes the systematic search of BO with the heuristic reasoning of LLMs.

The ADO-LLM Framework

ADO-LLM implements a cooperative optimization loop consisting of three primary components: an LLM Agent, a GP-BO Proposer, and a High-Quality Data Sampler.

1. The LLM Agent

Utilizing GPT-3.5 Turbo, the LLM Agent serves as the "expert designer." Unlike standard optimizers that process only numerical vectors, the LLM Agent employs In-Context Learning (ICL) and Chain-of-Thought (CoT) prompting. It analyzes the circuit netlist to identify component roles—such as identifying differential pairs or current mirrors—and suggests parameters based on established design principles, such as maintaining transistors in the saturation region.

2. The GP-BO Proposer

To ensure the search does not become trapped in the LLM's potential biases, a Gaussian Process-based Bayesian Optimization proposer acts as the "systematic explorer." It maintains a surrogate model of the design space and uses an acquisition function (q-Expected Improvement) to suggest diverse candidates, ensuring global exploration.

3. High-Quality Data Sampler

The sampler acts as a feedback bridge. It identifies the "top-k" performing designs from previous iterations and feeds them back to the LLM as demonstrations. This allows the LLM to learn the specific sensitivities of the current design task in real-time.

Experimental Methodology

The framework was evaluated using a commercial 90nm CMOS technology on two benchmark circuits:

  1. Two-Stage Differential Amplifier: Featuring 14 design parameters.
  2. Hysteresis Comparator: Featuring 12 design parameters.

The performance of ADO-LLM was benchmarked against standard GP-BO (with random initialization) and a standalone LLM-based optimizer. Evaluations were conducted using the HSPICE simulator to provide high-fidelity ground truth.

Key Insights

The "Warm Start" Advantage

One of the primary strengths of ADO-LLM is its ability to perform "zero-shot" initialization. By leveraging the LLM’s prior knowledge of typical transistor ratios, the optimization process begins with a high-quality starting point, significantly reducing the time spent in non-viable regions of the design space.

Qualitative Feedback Integration

While standard BO only receives a numerical Figure of Merit (FOM), ADO-LLM incorporates qualitative data. The simulator provides feedback on the operating region of transistors (e.g., linear vs. saturation). The LLM processes this information to apply engineering rules, such as adjusting bias voltages to ensure proper transistor operation, a task that purely numerical optimizers cannot perform natively.

Results

The experimental results demonstrate significant improvements in both efficiency and design quality:

  • Specification Satisfaction: In the differential amplifier benchmark, ADO-LLM was the only method to satisfy 100% of the design specifications (including Gain, CMRR, GBW, Phase Margin, and Power) within the allocated simulation budget.
  • Data Efficiency: ADO-LLM achieved a superior Figure of Merit (FOM) in just 20 iterations, whereas standard GP-BO required 80 iterations to reach a comparable level—representing a 4x improvement in data efficiency.
  • Optimization Robustness: For the hysteresis comparator, ADO-LLM reached a positive FOM (0.90), while both the standalone GP-BO and LLM methods failed to meet the minimum specifications, resulting in negative FOM scores.

Limitations and Future Directions

Despite its success, ADO-LLM faces challenges regarding its reliance on closed-source APIs, which introduces costs and limits transparency. Future research will explore the use of fine-tuned, domain-specific open-source models, such as "ChipNeMo." Additionally, as the number of parameters increases in complex System-on-Chip (SoC) designs, the authors propose a hierarchical optimization strategy where the LLM assists in decomposing large circuits into manageable sub-blocks.

Conclusion

ADO-LLM represents a significant step toward fully automated, physics-aware circuit design. By combining the rigorous exploration of Bayesian Optimization with the heuristic reasoning of Large Language Models, the framework overcomes the limitations of "black-box" optimization, offering a path toward more efficient and reliable semiconductor engineering.

Summary updated

BibTeX
@inproceedings{yin2024ado,
  author = {Yuxuan Yin and Yu Wang and Boxun Xu and Peng Li},
  title = {{ADO-LLM: Analog Design Bayesian Optimization with In-Context Learning of Large Language Models}},
  booktitle = {ICCAD},
  year = {2024},
  doi = {10.1145/3676536.3676816},
}
Figure