Multimodal Chip Physical Design Engineer Assistant

Yun-Da Tsai, Chang-Yu Chao, Liang-Yeh Shen, Tsung-Han Lin, Haoyu Yang, Mark Ho, Yi-Chen Lu, Wen-Hao Liu, Shou-De Lin, Haoxing RenarXiv 2025 · DOI · arXiv

Abstract

Routing congestion is a critical bottleneck in chip physical design, often requiring tedious manual iterations by engineers. We present a Multimodal Large Language Model Assistant (MLLMA) that predicts congestion and provides interpretable, actionable design suggestions. By integrating automated feature engineering via a "Genetic Instruct" strategy with a preference-learning framework, MLLMA achieves state-of-the-art accuracy on the CircuitNet benchmark and offers a "Design Suggestion Deck" to guide layout optimization.

Introduction

In modern VLSI design, routing congestion occurs when wires are too densely packed for manufacturing or performance constraints. Traditional Electronic Design Automation (EDA) tools are often slow and act as "black boxes," flagging issues without explaining their root causes (e.g., macro placement vs. pin density). While AI models have accelerated prediction, they typically lack the interpretability required for actionable engineering guidance. MLLMA addresses this by bridging the gap between raw spatial prediction and human-readable design advice.

Methodology

The MLLMA framework operates in two primary stages: automated feature discovery and interpretable inference.

Automated Feature Engineering (Genetic Instruct)

To avoid the limitations of manually defined metrics, the system employs MLLM agents to generate Python scripts for feature extraction from layout images (such as RUDY maps). Using a "Genetic Instruct" approach—incorporating crossover and mutation—the agents evolve a pool of physically meaningful features. These features are then ranked and pruned using a Random Forest model to ensure only high-impact metrics are retained.

Interpretable Preference Learning

The core architecture utilizes a multimodal backbone (MiniCPM) to process layout images alongside text-based configuration logs. A specialized gating layer learns to weight these features based on the specific design context, allowing the model to identify which factors (e.g., macro density vs. pin clustering) are driving congestion in a particular layout.

The MLLMA architecture integrates a Vision Encoder and MLLM backbone with an Interpretable Features Fusion module to predict congestion scores and generate design suggestions.

Dataset and Evaluation

The model was evaluated using CircuitNet, a large-scale open-source benchmark containing over 10,000 synthesized chip designs (CPUs, GPUs, and AI accelerators) fabricated on a 14nm FinFET process. The dataset includes diverse layout representations, including routability, timing, and power maps, as well as detailed configuration logs.

Results

MLLMA demonstrates superior performance over established baselines like GPDL and CircuitFormer. Quantitatively, it achieved a Structural Similarity Index (SSIM) of 0.807 and a low NRMSE of 0.045.

Qualitatively, the model excels at localizing congestion "hotspots" that other models frequently overlook. Beyond prediction, the model's "Design Suggestion Deck" was validated through a case study where adjusting a design based on the top five suggested optimizations led to a measurable reduction in actual routing congestion.

Qualitative comparison of congestion map predictions: MLLMA (Ours) more accurately identifies hotspots (red boxes) compared to the GPDL baseline when measured against the ground truth.

Discussion and Future Work

A key insight from this research is the utility of MLLMs as coding agents to discover complex spatial metrics humans might overlook. Furthermore, the context-aware gating mechanism confirms that feature importance is not static; for instance, macro_density may be critical in one design while pin_clustering dominates another.

Future iterations of MLLMA aim to expand beyond congestion to include Power, Performance, and Area (PPA) metrics, such as IR drop and timing closure. Additionally, the researchers plan to develop a more interactive, multi-turn dialogue interface, allowing engineers to refine designs iteratively through natural language conversation with the assistant.

Summary updated

Abstract

Modern chip physical design relies heavily on Electronic Design Automation (EDA) tools, which often struggle to provide interpretable feedback or actionable guidance for improving routing congestion. In this work, we introduce a Multimodal Large Language Model Assistant (MLLMA) that bridges this gap by not only predicting congestion but also delivering human-interpretable design suggestions. Our method combines automated feature generation through MLLM-guided genetic prompting with an interpretable preference learning framework that models congestion-relevant tradeoffs across visual, tabular, and textual inputs. We compile these insights into a "Design Suggestion Deck" that surfaces the most influential layout features and proposes targeted optimizations. Experiments on the CircuitNet benchmark demonstrate that our approach outperforms existing models on both accuracy and explainability. Additionally, our design suggestion guidance case study and qualitative analyses confirm that the learned preferences align with real-world design principles and are actionable for engineers. This work highlights the potential of MLLMs as interactive assistants for interpretable and context-aware physical design optimization.
BibTeX
@misc{tsai2025multimodal,
  author = {Yun-Da Tsai and Chang-Yu Chao and Liang-Yeh Shen and Tsung-Han Lin and Haoyu Yang and Mark Ho and Yi-Chen Lu and Wen-Hao Liu and Shou-De Lin and Haoxing Ren},
  title = {{Multimodal Chip Physical Design Engineer Assistant}},
  year = {2025},
  eprint = {2510.15872},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2510.15872},
}
Figure