Abstract
Placement is a critical phase in modern chip design, requiring the arrangement of millions of circuit modules on a 2D canvas to optimize for delay and energy consumption. While deep reinforcement learning (RL) has emerged as a potential autonomous solution to replace labor-intensive human-centric workflows, it is often hindered by massive design spaces and representation gaps. This paper presents MaskPlace, an RL framework that recasts placement as a pixel-level visual representation learning task. By utilizing a high-resolution canvas and a dense reward function, MaskPlace generates valid chip layouts in hours. Experimental results across 24 public benchmarks demonstrate that MaskPlace achieves 60%–90% wirelength reduction compared to previous RL methods and consistently guarantees zero overlaps, outperforming both graph-based RL approaches and traditional optimization tools.
Introduction
In the physical design of integrated circuits, placement determines the exact coordinates of macros (e.g., memory blocks) and standard cells (e.g., logic gates). The complexity of this task is immense, with design spaces reaching orders of magnitude that challenge traditional optimization algorithms. Historically, this process has relied on human experts and commercial Electronic Design Automation (EDA) tools, often taking months to reach a routable solution.
Recent attempts to use Reinforcement Learning for placement have faced three primary obstacles: the representation gap, where hypergraph models ignore physical pin offsets; sparse rewards, where feedback is only provided at the end of a long placement sequence; and hard constraints, specifically the difficulty of preventing module overlaps without destructive post-processing legalization steps.
Methodology: Placement as Visual Learning
MaskPlace shifts the paradigm from graph-based learning to pixel-level visual representation learning. By treating the chip canvas as an image rather than a mathematical graph, the model leverages the spatial inductive biases of convolutional neural networks (CNNs).
The Triple-Mask Representation
To capture the complex constraints of the placement problem, MaskPlace represents the chip state using three distinct 224x224 feature maps:
- Position Mask: A binary grid that identifies feasible placement areas. This mask is updated dynamically to exclude regions where a module would overlap with previously placed components, effectively embedding the "zero-overlap" constraint directly into the agent's action space.
- Wire Mask: A heatmap representing the potential Half-Perimeter Wirelength (HPWL) for every possible coordinate of the current module. Unlike graph-based methods, this mask accounts for specific pin offsets—the exact points on a module where wires connect—leading to much more accurate wirelength estimations.
- View Mask: A global spatial observation that provides the agent with a comprehensive view of the current layout progress.
Architecture and Reward Logic
The framework employs a CNN-based architecture, utilizing a ResNet-18 encoder to process the masks and a deconvolutional decoder to map features back to the action space.
Crucially, MaskPlace addresses the "sparse reward" problem by implementing a dense reward function. The agent receives immediate feedback after every module placement based on the change in total wirelength. Additionally, a congestion satisfaction block ensures that the agent avoids creating overly dense areas that would be impossible to route in later design stages.
Experimental Evaluation
MaskPlace was evaluated on 24 public benchmarks, including the ISPD2005 suite, the IBM Benchmark Suite, and the Ariane RISC-V CPU. This represents one of the most extensive evaluations in the RL-based placement literature, covering circuits with over 1,200 macros and 1 million standard cells.
Key Results
- Wirelength Optimization: MaskPlace achieved a 60% to 90% reduction in wirelength compared to DeepPR. When compared to Google’s "Graph Placement" method, MaskPlace showed up to a 9x improvement in specific metrics.
- Zero Overlap Guarantee: MaskPlace is the first RL-based method to consistently achieve 0% overlap across all benchmarks. This eliminates the need for a separate "legalization" step, which often degrades the quality of optimization-based layouts.
- Robustness and Reliability: On complex benchmarks like "ariane" and "bigblue3," traditional tools such as NTUPlace3 and DREAMPlace frequently failed to produce valid, legal layouts. MaskPlace succeeded in these scenarios, producing high-quality, routable designs.
- Efficiency: The system can generate a high-quality layout within a few hours, making it a viable competitor to commercial EDA tools in fast-paced design cycles.
Discussion and Future Work
The success of MaskPlace highlights that chip placement is fundamentally a spatial and visual problem. By "baking in" constraints via the Position Mask and providing dense feedback, the model overcomes the primary limitations of previous RL attempts.
Currently, MaskPlace is optimized for macro placement, with standard cells handled by existing tools like DREAMPlace in a subsequent step. Future research will explore scaling these visual RL techniques to handle the simultaneous placement of millions of standard cells and integrating other design stages—such as routing and timing analysis—into a unified, end-to-end visual learning framework.
Conclusion
MaskPlace demonstrates that visual representation learning is a powerful alternative to hypergraph-based approaches for chip design. By achieving superior wirelength, ensuring zero overlaps, and maintaining high computational efficiency, it sets a new state-of-the-art for AI-assisted physical design.
Summary updated