Pretraining Graph Neural Networks for few-shot Analog Circuit Modeling and Design

Kourosh Hakhamaneshi, Marcel Nassar, Mariano Phielipp, Pieter Abbeel, Vladimir Stojanovic

Abstract

Being able to predict the performance of circuits without running expensive simulations is a desired capability that can catalyze automated design. In this paper, we present a supervised pretraining approach to learn circuit representations that can be adapted to new circuit topologies or unseen prediction tasks. We hypothesize that if we train a neural network (NN) that can predict the output DC voltages of a wide range of circuit instances it will be forced to learn generalizable knowledge about the role of each circuit element and how they interact with each other. The dataset for this supervised learning objective can be easily collected at scale since the required DC simulation to get ground truth labels is relatively cheap. This representation would then be helpful for few-shot generalization to unseen circuit metrics that require more time consuming simulations for obtaining the ground-truth labels. To cope with the variable topological structure of different circuits we describe each circuit as a graph and use graph neural networks (GNNs) to learn node embeddings. We show that pretraining GNNs on prediction of output node voltages can encourage learning representations that can be adapted to new unseen topologies or prediction of new circuit level properties with up to 10x more sample efficiency compared to a randomly initialized model. We further show that we can improve sample efficiency of prior SoTA model-based optimization methods by 2x (almost as good as using an oracle model) via fine-tuning pretrained GNNs as the feature extractor of the learned models.

Introduction

Analog circuit design automation is hindered by the sensitivity of circuit performance to component sizing and topology. While Machine Learning (ML) offers a potential solution for performance prediction, it is often limited by data scarcity due to the high computational cost of Spice simulations. Furthermore, traditional ML models lack generalizability across different circuit topologies. This paper addresses these challenges by proposing a supervised pretraining framework for Graph Neural Networks (GNNs) that learns generalizable circuit representations.

Methods

The proposed framework introduces a novel graph representation and a pretraining task designed to capture circuit physics.

Terminal-Explicit Graph Representation

Standard methods often map entire devices to single nodes, leading to ambiguity at connection points. We propose a "terminal-explicit" graph where each device terminal (e.g., drain, gate, source) and each net (wire) is represented as a distinct node. This approach preserves the exact connectivity and physical structure of the circuit.

Terminal-explicit graph representation vs. prior work.

DC Voltage Pretraining Task

We utilize the prediction of DC node voltages (bias points) as a pretraining task. DC simulations are computationally inexpensive compared to transient or AC simulations. Training a GNN to predict these voltages forces the model to learn how components interact and how signals propagate through the topology, effectively learning the "grammar" of analog electronics.

Model Architecture

The architecture employs a DeepGEN GNN backbone to generate contextualized node features. For circuit-level predictions, such as total gain or bandwidth, we use a learnable attention-based aggregation layer inspired by Perceiver IO. This allows the model to dynamically attend to the most relevant nodes in the circuit.

GNN architecture with attention-based aggregation.

Experimental Results

The model was evaluated on synthetic Resistor Ladder datasets and a large-scale Operational Amplifier (OpAmp) dataset containing 126,000 designs.

  • Sample Efficiency: When fine-tuning on new tasks like OpAmp gain prediction, the pretrained model achieved the same accuracy as a randomly initialized model using 10x fewer samples.
  • Zero-Shot Generalization: Models pretrained on small resistor ladders (10 branches) successfully predicted voltages on much larger, unseen ladders (up to 128 branches) without further training, indicating an implicit understanding of Kirchhoff’s laws.
  • Optimization Boost: Integrating the pretrained GNN into the BagNet optimizer resulted in a 2x improvement in sample efficiency, performing nearly as well as an oracle model.

Conclusion

Pretraining GNNs on node voltage prediction provides a robust foundation for analog circuit modeling. This approach significantly improves sample efficiency and enables generalization across diverse topologies. Future work will explore extending this framework to transient analysis and a broader range of circuit families.

Summary updated

Abstract

Being able to predict the performance of circuits without running expensive simulations is a desired capability that can catalyze automated design. In this paper, we present a supervised pretraining approach to learn circuit representations that can be adapted to new circuit topologies or unseen prediction tasks. We hypothesize that if we train a neural network (NN) that can predict the output DC voltages of a wide range of circuit instances it will be forced to learn generalizable knowledge about the role of each circuit element and how they interact with each other. The dataset for this supervised learning objective can be easily collected at scale since the required DC simulation to get ground truth labels is relatively cheap. This representation would then be helpful for few-shot generalization to unseen circuit metrics that require more time consuming simulations for obtaining the ground-truth labels. To cope with the variable topological structure of different circuits we describe each circuit as a graph and use graph neural networks (GNNs) to learn node embeddings. We show that pretraining GNNs on prediction of output node voltages can encourage learning representations that can be adapted to new unseen topologies or prediction of new circuit level properties with up to 10x more sample efficiency compared to a randomly initialized model. We further show that we can improve sample efficiency of prior SoTA model-based optimization methods by 2x (almost as good as using an oracle model) via fintuning pretrained GNNs as the feature extractor of the learned models.
BibTeX
@article{hakhamaneshi2023pretraining,
  author = {Kourosh Hakhamaneshi and Marcel Nassar and Mariano Phielipp and Pieter Abbeel and Vladimir Stojanovic},
  title = {{Pretraining Graph Neural Networks for few-shot Analog Circuit Modeling and Design}},
  journal = {IEEE Trans. Comput. Aided Des. Integr. Circuits Syst.},
  year = {2023},
  url = {https://openreview.net/forum?id=muKpu4oPUE},
}
Figure