Routing and Placement of Macros using Deep Reinforcement Learning

Mrinal MathurarXiv 2022 · OpenReview

Abstract

Chip placement has been one of the most time-consuming tasks in semiconductor design. Negligence in this area often leads to project delays and market shortages. An engineer placing macros on a chip must optimize for power, performance, and area (PPA). To address these challenges, we introduce a new method using Reinforcement Learning (RL) to train an agent to place chip netlist nodes onto a canvas. We build a neural architecture that accurately rewards the agent across a wide variety of input netlists.

Introduction

As semiconductor technology advances toward the 3nm node, the complexity of chip design has outpaced traditional scaling. A major bottleneck is chip placement: determining the optimal locations for millions of gates and macros (large functional blocks). Traditional manual placement is slow, taking weeks of human effort, while automated analytic solvers often struggle to balance wirelength, power consumption, and routing congestion. This research proposes a learning-based agent capable of "understanding" netlist connections and placing components optimally in a fraction of the time.

Methodology

The chip placement problem is framed as a Markov Decision Process (MDP) and solved using Proximal Policy Optimization (PPO).

State Representation and Architecture

To capture the complex topological features of a netlist, the system uses GraphSAGE (a Graph Neural Network) to generate embeddings. These embeddings represent node "closeness" and spectral characteristics, such as the Fiedler value. The architecture also incorporates a pre-trained ConvNet to extract features from 2D layout images and Transformers to process sequential data.

The comprehensive neural architecture integrates GNNs for netlist embeddings, ConvNets for layout features, and sequential models to drive policy and value networks.

Action and Reward

The action space is designed to prioritize macro functionality first, effectively reducing the search space. The reward function ($R_t$) is defined as the negative sum of Half-Perimeter Wirelength (HPWL) and congestion. By minimizing this value, the agent learns to produce shorter wires and less crowded layouts.

Experimental Setup

The model was evaluated using high-performance benchmarks:

  • RISC-V (ARIANE): A 64-bit CPU core.
  • Qualcomm Multi-core Benchmarks: Proprietary designs with over 60,000 macros per core.
  • Technology Node: Netlists were synthesized for the TSMC 29nm node using industry-standard tools like Synopsis Design Compiler and Cadence Innovus.

Results and Discussion

The RL-based approach significantly outperformed both human engineers and the automated tool RePlAce across key metrics.

Method Timing (ps) Area ($\mu m^2$) Power (W) Wirelength (m)
Manual 233.2 169,341 3.24 52.14
RePlAce 87.9 168,724 3.73 51.11
Ours (RL) 52.0 168,645 3.23 51.09

The RL agent achieved a timing of 52ps, a 40% improvement over RePlAce, while also reducing wirelength and power consumption.

Visual comparison showing the RL method's placement versus a human engineer's work after four days. The RL approach achieves superior density and routing potential in significantly less time.

Key Insights

  • Spectral Intuition: Using the Fiedler value allows the agent to mathematically anticipate congestion.
  • Functionality-First: Grouping related logic levels mimics human expertise, leading to more logical floorplans.
  • Zero-Shot Potential: Once trained, the policy can potentially be applied to new netlists with minimal fine-tuning.

Conclusion

While the initial training cost is high (approximately 100 hours on a high-performance cluster), the resulting model generates high-quality placements in hours rather than days. Future work will focus on large-scale pre-training to enhance zero-shot generalization across diverse chip architectures.

Summary updated

Abstract

Chip placement has been one of the most time consuming task in any semi conductor area, Due to this negligence, many projects are pushed and chips availability in real markets get delayed. An engineer placing macros on a chip also needs to place it optimally to reduce the three important factors like power, performance and time. Looking at these prior problems we wanted to introduce a new method using Reinforcement Learning where we train the model to place the nodes of a chip netlist onto a chip canvas. We want to build a neural architecture that will accurately reward the agent across a wide variety of input netlist correctly.
BibTeX
@misc{mathur2022routing,
  author = {Mrinal Mathur},
  title = {{Routing and Placement of Macros using Deep Reinforcement Learning}},
  year = {2022},
  eprint = {2205.09289},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2205.09289},
}
Figure