Transformations of High-Level Synthesis Codes for High-Performance Computing

Johannes de Fine Licht, Maciej Besta, Simon Meierhans, Torsten Hoefler

This paper, authored by researchers at ETH Zurich, serves as a definitive "cheat sheet" for performance engineers looking to bridge the gap between high-level software and high-performance hardware. It systematizes the transformations necessary to turn naive C++ or OpenCL code into highly optimized spatial architectures for FPGAs.

TL;DR

The authors present a comprehensive toolbox of code transformations for High-Level Synthesis (HLS) that enable software-defined hardware to reach its full performance potential. By applying these hardware-aware optimizations to standard kernels, they demonstrate dramatic cumulative speedups of up to 29,950x over naive implementations.

What Problem Does This Paper Address?

As Dennard scaling ends, traditional "load/store" architectures (CPUs and GPUs) are increasingly limited by power consumption and the high cost of data movement. Spatial computing—specifically FPGAs—offers a path toward extreme energy efficiency by allowing developers to tailor the hardware layout to the specific dataflow of an application.

However, there is a massive "productivity-performance gap." While HLS tools allow developers to write hardware using procedural languages like C++, "naive" software code performs abysmally on FPGAs. Traditional software optimizations (like those for CPUs) are insufficient because they don't account for hardware-specific constraints like deep pipelining, distributed on-chip memory, and physical routing. This paper provides the missing manual for optimizing HLS code for High-Performance Computing (HPC).

Methods

The authors categorize HLS transformations into three primary pillars:

  1. Pipelining Transformations: These focus on achieving an Initiation Interval (II) of 1, meaning the hardware accepts a new input every single clock cycle. Techniques include accumulation interleaving (to resolve loop-carried dependencies) and loop fusion/flattening (to eliminate pipeline "drain" time).
  2. Scaling Transformations: These increase parallelism. The authors distinguish between Horizontal Unrolling (vectorization/SIMD-style) and Vertical Unrolling. Vertical unrolling is particularly novel for software engineers; it involves replicating compute units to exploit temporal locality, allowing the system to scale performance without demanding more off-chip memory bandwidth.
  3. Memory Transformations: These optimize how data moves between off-chip DRAM and on-chip logic. Key methods include memory access extraction (separating I/O from computation), striping (distributing data across multiple memory banks), and type demotion (reducing bit-width to save resources).

The authors provide a "cheat sheet" (Table 1 in the paper) that maps specific performance bottlenecks to the corresponding transformation.

Datasets

To validate their toolbox, the authors optimized three classic HPC kernels, providing the source code as an open-source reference:

  • 2D Stencil Code: A 4-point stencil representing partial differential equation solvers.
  • General Matrix Multiplication (GEMM): The backbone of linear algebra and deep learning.
  • N-Body Problem: A simulation of dynamical systems of particles.

Evaluation was performed on a Xilinx Alveo U250 board, a high-end FPGA used in data centers.

Key Insights and Ideas

  • The "II=1" Mantra: In HLS, the most critical metric isn't just clock speed, but the Initiation Interval. A pipeline that stalls every other cycle (II=2) immediately cuts throughput by 50%.
  • Spatial vs. Temporal Locality: While CPUs use caches to manage temporal locality, HLS allows developers to build "application-specific dataflow." This means using shift registers and line buffers to keep data moving through the chip rather than constantly fetching it from memory.
  • Vertical Unrolling is the "Secret Sauce": For many HPC applications, memory bandwidth is the bottleneck. Vertical unrolling allows a design to scale by reusing the same data across multiple processing elements (PEs) arranged in a systolic array, effectively getting "free" computation for every byte read.
  • Software Optimizations Still Matter: Interestingly, many classic CPU optimizations (like constant folding or dead-code elimination) are still vital in HLS because they prevent the synthesis tool from wasting precious hardware gates on unnecessary logic.

Key Results

The performance gains from applying these transformations sequentially are staggering:

  • Stencil Code: Jumped from <0.1 GOp/s (naive) to 409.3 GOp/s, a cumulative speedup of 18,270x.
  • Matrix Multiplication: Scaled from <0.1 GOp/s to 497.0 GOp/s, a cumulative speedup of 29,950x.
  • N-Body: Improved from 1.6 GOp/s to 270.7 GOp/s, a 167x speedup.

The results show that while HLS tools provide the path to hardware design, the performance is entirely dependent on the developer's ability to apply these hardware-aware transformations.

Limitations and Future Work

  • Manual Effort: Most of these transformations currently require manual code restructuring. While some HLS compilers are getting better at auto-resolving simple patterns, complex optimizations like vertical unrolling still require human "performance engineering."
  • Place and Route Bottlenecks: As designs become more parallel, the physical wiring (routing) on the FPGA becomes a bottleneck. The authors note that code-level transformations can only do so much if the chip's physical layout becomes too congested.
  • Toolchain Variance: While the principles are universal, the specific pragmas and "compiler hints" vary significantly between Xilinx (Vivado/Vitis) and Intel (OpenCL) toolflows. The authors hope their work encourages more standardized, hardware-aware optimization passes in future HLS compilers.

Summary updated

Abstract

Spatial computing architectures promise a major stride in performance and energy efficiency over the traditional load/store devices currently employed in large scale computing systems. The adoption of high-level synthesis (HLS) from languages such as C++ and OpenCL has greatly increased programmer productivity when designing for such platforms. While this has enabled a wider audience to target spatial computing architectures, the optimization principles known from traditional software design are no longer sufficient to implement high-performance codes, due to fundamentally distinct aspects of hardware design, such as programming for deep pipelines, distributed memory resources, and scalable routing. To alleviate this, we present a collection of optimizing transformations for HLS, targeting scalable and efficient architectures for high-performance computing (HPC) applications. We systematically identify classes of transformations (pipelining, scalability, and memory), the characteristics of their effect on the HLS code and the resulting hardware (e.g., increasing data reuse or resource consumption), and the objectives that each transformation can target (e.g., resolve interface contention, or increase parallelism). We show how these can be used to efficiently exploit pipelining, on-chip distributed fast memory, and on-chip dataflow, allowing for massively parallel architectures. To quantify the effect of various transformations, we cover the optimization process of a sample set of HPC kernels, provided as open source reference codes. We aim to establish a common toolbox to guide both performance engineers and compiler engineers in tapping into the performance potential offered by spatial computing architectures using HLS.
BibTeX
@article{licht2021transformations,
  author = {Johannes de Fine Licht and Maciej Besta and Simon Meierhans and Torsten Hoefler},
  title = {{Transformations of High-Level Synthesis Codes for High-Performance Computing}},
  journal = {IEEE Trans. Parallel Distributed Syst.},
  year = {2021},
  url = {https://openreview.net/forum?id=isidZPwvL5},
}

Loading related content...

Figure