Abstract: Spatial computing architectures promise a major stride in performance and energy efficiency over the traditional load/store devices currently employed in large scale computing systems. The adoption of high-level synthesis (HLS) from languages such as C++ and OpenCL has greatly increased programmer productivity when designing for such platforms. While this has enabled a wider audience to target spatial computing architectures, the optimization principles known from traditional software design are no longer sufficient to implement high-performance codes, due to fundamentally distinct aspects of hardware design, such as programming for deep pipelines, distributed memory resources, and scalable routing. To alleviate this, we present a collection of optimizing transformations for HLS, targeting scalable and efficient architectures for high-performance computing (HPC) applications. We systematically identify classes of transformations (pipelining, scalability, and memory), the characteristics of their effect on the HLS code and the resulting hardware, and the objectives that each transformation can target. We show how these can be used to efficiently exploit pipelining, on-chip distributed fast memory, and on-chip dataflow, allowing for massively parallel architectures. To quantify the effect of various transformations, we cover the optimization process of a sample set of HPC kernels, provided as open source reference codes.
Introduction
As Dennard scaling ends, traditional "load/store" architectures like CPUs and GPUs are increasingly limited by power consumption and the high cost of data movement. Spatial computing—specifically FPGAs—offers a path toward extreme energy efficiency by allowing developers to tailor the hardware layout to the specific dataflow of an application. However, a significant "productivity-performance gap" exists. While HLS tools allow developers to write hardware using procedural languages like C++, naive software code performs poorly on FPGAs. Traditional software optimizations are insufficient because they do not account for hardware-specific constraints like deep pipelining and distributed on-chip memory. This work provides a systematic toolbox for optimizing HLS code for High-Performance Computing (HPC).
Core HLS Transformations
The authors categorize HLS transformations into three primary pillars: pipelining, scaling, and memory management.
Pipelining Transformations
The primary goal in HLS is achieving an Initiation Interval (II) of 1, meaning the hardware accepts a new input every clock cycle. A major obstacle to this is the loop-carried dependency, where a calculation depends on the result of a previous iteration. If the operation (e.g., a floating-point addition) takes multiple cycles ($L_{acc}$), the pipeline must stall, leading to an $II > 1$.
To resolve this, developers can use accumulation interleaving. By interleaving $M$ independent accumulation operations (where $M \ge L_{acc}$), the pipeline can remain full, effectively hiding the operation latency and restoring $II=1$ throughput.
Scaling and Memory Transformations
Scaling transformations increase parallelism through Horizontal Unrolling (SIMD-style vectorization) and Vertical Unrolling. Vertical unrolling is particularly powerful for HPC; it involves replicating compute units to exploit temporal locality. This allows the system to scale performance by reusing data across multiple processing elements (PEs) in a systolic array without increasing off-chip memory bandwidth requirements.
Memory transformations focus on optimizing data movement between off-chip DRAM and on-chip logic. Key methods include memory access extraction (separating I/O from computation), striping (distributing data across multiple memory banks), and type demotion (reducing bit-width to save hardware resources).
Bridging the Productivity Gap
Implementing these transformations requires a shift in how software engineers approach code. For example, the MultiSolver pattern shown below demonstrates how to structure a loop to interleave multiple independent problems, ensuring that the hardware pipeline stays busy even when individual operations have high latency.
Results and Performance Impact
To validate this toolbox, the authors optimized three classic HPC kernels on a Xilinx Alveo U250 board. The performance gains from applying these transformations sequentially are massive:
- 2D Stencil Code: Improved from <0.1 GOp/s (naive) to 409.3 GOp/s, a cumulative speedup of 18,270x.
- General Matrix Multiplication (GEMM): Scaled from <0.1 GOp/s to 497.0 GOp/s, a cumulative speedup of 29,950x.
- N-Body Problem: Improved from 1.6 GOp/s to 270.7 GOp/s, a 167x speedup.
Conclusion
While HLS tools provide a productive path to hardware design, reaching peak performance requires deep hardware awareness. By systematizing these transformations, we aim to establish a common toolbox that allows performance engineers to tap into the full potential of spatial computing architectures. Future work will focus on automating these complex patterns within HLS compilers to further reduce the manual engineering effort required.
Summary updated