Learning to Compare Hardware Designs for High-Level Synthesis

Yunsheng Bai, Atefeh Sohrabizadeh, Zijian Ding, Rongjian Liang, Weikai Li, Ding Wang, Haoxing Ren, Yizhou Sun, Jason Cong

Abstract

High-level synthesis (HLS) is an automated design process that transforms high-level code into hardware designs, enabling the rapid development of hardware accelerators. HLS relies on pragmas, which are directives inserted into the source code to guide the synthesis process, and pragmas have various settings and values that significantly impact the resulting hardware design. State-of-the-art ML-based HLS methods, such as HARP, first train a deep learning model, typically based on graph neural networks (GNNs) applied to graph-based representations of the source code and pragmas. They then perform design space exploration (DSE) to explore the pragma design space, rank candidate designs using the model, and return the top designs. However, traditional DSE methods face challenges due to the highly nonlinear relationship between pragma settings and performance metrics, along with complex interactions between pragmas that affect performance in non-obvious ways. To address these challenges, we propose compareXplore, a novel approach that learns to compare hardware designs for effective HLS optimization. CompareXplore introduces a hybrid loss function that combines pairwise preference learning with pointwise performance prediction, enabling the model to capture both relative preferences and absolute performance. Moreover, we introduce a novel node difference attention module that focuses on the most informative differences between designs, enabling the model to identify critical pragmas impacting performance. CompareXplore adopts a two-stage DSE, where a pointwise prediction model is used for the initial design pruning, followed by a pairwise comparison stage for precise performance verification. In extensive experiments, compareXplore achieves significant improvements in ranking metrics and generates high-quality HLS results for the selected designs, outperforming the existing SOTA method.

Introduction

High-level synthesis (HLS) has revolutionized hardware design by allowing developers to transform high-level programming languages like C++ into hardware descriptions (RTL). This process relies heavily on pragmas—directives that guide the compiler on how to optimize loops, manage memory, and implement parallelism. However, the design space created by these pragmas is vast and highly non-linear. A single change in a pragma setting can have cascading effects on performance, making it difficult for traditional machine learning models to accurately predict hardware metrics.

Current state-of-the-art methods, such as HARP, typically employ a "pointwise" approach: they train Graph Neural Networks (GNNs) to predict the absolute performance (e.g., latency) of a single design. We argue that this task is unnecessarily difficult. For the purpose of design space exploration (DSE), an engineer doesn't necessarily need to know the exact latency of every candidate; they simply need to know if Design A is better than Design B.

The compareXplore Framework

To address these challenges, we introduce compareXplore, a framework that shifts the focus from absolute performance prediction to comparative learning. By teaching models how to compare two designs, we can more effectively navigate the complex HLS design space.

Node Difference Attention

A core innovation of compareXplore is the Node Difference Attention module. When comparing two designs, the model does not analyze them in isolation. Instead, it calculates the differences between the GNN embeddings of the two designs. This attention mechanism allows the model to "zoom in" on the specific pragma modifications that drive performance changes, effectively ignoring the parts of the code that remain constant.

Hybrid Loss Function

To ensure the model retains a sense of scale while mastering relative rankings, we utilize a hybrid loss function:

  1. Pointwise Loss (MSE): Maintains the model's ability to estimate absolute performance.
  2. Pairwise Loss (Cross-Entropy): Optimizes the model's ability to correctly rank one design over another.

Our experiments indicate that a balanced weight ($\alpha=1$) between these two losses provides the optimal performance in ranking accuracy, measured by Kendall’s $\tau$.

Two-Stage Design Space Exploration

Navigating a massive design space requires efficiency. compareXplore employs a two-stage DSE strategy:

  1. Stage 1: Pruning. A fast pointwise model quickly evaluates the design space and discards low-performing candidates, narrowing the field to a set of top $K_1$ designs.
  2. Stage 2: Verification. The remaining candidates undergo a rigorous pairwise comparison. Inspired by Ranked Choice Voting (RCV), the model iteratively compares designs and eliminates the least preferred options until the optimal designs are identified. This stage is computationally efficient, accounting for only 12.94% of the total exploration time.

Experimental Evaluation

We evaluated compareXplore using a dataset of 10,868 unique hardware designs across 40 computational kernels (e.g., image and signal processing), synthesized using Xilinx Vitis HLS.

Performance Gains

The comparative approach yielded significant improvements over the previous state-of-the-art (HARP):

  • Latency Reduction: compareXplore achieved an average 16.11% reduction in latency across all kernels.
  • Maximum Efficiency: In specific instances, such as the "adi" kernel, the framework reduced latency by nearly 50%.
  • Ranking Accuracy: The model achieved a higher Kendall’s $\tau$ score (0.4319 vs 0.4157), indicating a much stronger alignment with true hardware performance rankings.

Discussion: Comparison vs. Prediction

An intriguing finding from our research is that compareXplore actually exhibited a higher Root Mean Square Error (RMSE) in absolute latency prediction than the baseline HARP. Despite this, it consistently found superior hardware designs. This confirms our hypothesis: a model does not need to be a perfect calculator to be an excellent scout. By focusing on the relative impact of pragma changes, the model becomes more robust to the non-linearities of the HLS process.

Limitations and Future Directions

While compareXplore excels at ranking, its lower accuracy in absolute regression may be a drawback for engineers who require precise performance estimates before synthesis. Additionally, the model still struggles to distinguish between designs with very subtle pragma differences.

Future work will explore curriculum learning to help the model master these "hard comparisons." We also envision integrating compareXplore with Large Language Models (LLMs) to create a more intuitive, natural-language-driven interface for hardware optimization.

Summary updated

Abstract

High-level synthesis (HLS) is an automated design process that transforms high-level code into hardware designs, enabling the rapid development of hardware accelerators. HLS relies on pragmas, which are directives inserted into the source code to guide the synthesis process, and pragmas have various settings and values that significantly impact the resulting hardware design. State-of-the-art ML-based HLS methods, such as HARP, first train a deep learning model, typically based on graph neural networks (GNNs) applied to graph-based representations of the source code and pragmas. They then perform design space exploration (DSE) to explore the pragma design space, rank candidate designs using the model, and return the top designs. However, traditional DSE methods face challenges due to the highly nonlinear relationship between pragma settings and performance metrics, along with complex interactions between pragmas that affect performance in non-obvious ways. To address these challenges, we propose compareXplore, a novel approach that learns to compare hardware designs for effective HLS optimization. CompareXplore introduces a hybrid loss function that combines pairwise preference learning with pointwise performance prediction, enabling the model to capture both relative preferences and absolute performance. Moreover, we introduce a novel node difference attention module that focuses on the most informative differences between designs, enabling the model to identify critical pragmas impacting performance. CompareXplore adopts a two-stage DSE, where a pointwise prediction model is used for the initial design pruning, followed by a pairwise comparison stage for precise performance verification. In extensive experiments, compareXplore achieves significant improvements in ranking metrics and generates high-quality HLS results for the selected designs, outperforming the existing SOTA method.
BibTeX
@inproceedings{bai2024learning,
  author = {Yunsheng Bai and Atefeh Sohrabizadeh and Zijian Ding and Rongjian Liang and Weikai Li and Ding Wang and Haoxing Ren and Yizhou Sun and Jason Cong},
  title = {{Learning to Compare Hardware Designs for High-Level Synthesis}},
  booktitle = {MLCAD},
  year = {2024},
  doi = {10.1145/3670474.3685940},
}
Figure