HitGNN: High-Throughput GNN Training Framework on CPU+Multi-FPGA Heterogeneous Platform

Yi-Chien Lin, Bingyi Zhang, Viktor K. Prasanna

TL;DR

HitGNN is a novel framework that automates the deployment of Graph Neural Network (GNN) training onto heterogeneous CPU+Multi-FPGA platforms. By combining high-level APIs with automated hardware generation and smart workload balancing, it achieves up to 4.26× speedup and 27.21× better bandwidth efficiency compared to state-of-the-art multi-GPU systems.

What Problem Does This Paper Address?

Training Graph Neural Networks (GNNs) on massive real-world graphs is a computational nightmare. While GPUs are the standard workhorse, they often struggle with the irregular, "sparse" memory access patterns inherent in graph data, leading to poor memory bandwidth utilization.

FPGAs (Field-Programmable Gate Arrays) offer a promising alternative because their hardware can be customized to match specific data access patterns. However, using them is notoriously difficult:

  1. Hardware Expertise: Designing optimized kernels requires deep knowledge of hardware description languages.
  2. Scalability: Moving from one FPGA to many introduces massive headaches regarding workload imbalance (some chips finishing faster than others) and communication overhead between devices.
  3. Development Effort: Manually mapping complex training algorithms like DistDGL or PaGraph to hardware is time-prohibitive for most researchers.

HitGNN aims to bridge this gap, allowing users to describe GNN models in a few lines of code while the framework handles the "heavy lifting" of hardware optimization and multi-chip scaling.

Methods

The HitGNN framework consists of three main pillars:

  • High-Level APIs: Users define their GNN model (e.g., GCN, GraphSAGE) and training algorithm using Python-like APIs. This abstracts away the hardware complexity.
  • Automated Generators:
    • A Software Generator creates the host program to manage mini-batch sampling and CPU-FPGA communication.
  • An Accelerator Generator uses a Design Space Exploration (DSE) engine to automatically determine the best hardware parameters (like the number of processing elements) based on the specific FPGA's resources (DSPs, LUTs).
  • Key Optimizations for Multi-FPGA:
    • Two-Stage Task Scheduling: To prevent some FPGAs from sitting idle while others are still working on large partitions, HitGNN implements a scheduler that dynamically redistributes extra mini-batches to idle chips.
  • Direct CPU Data Fetching: Instead of complex and slow FPGA-to-FPGA communication, HitGNN leverages the host CPU's large memory as a central hub, allowing FPGAs to pull missing vertex features directly via PCIe, significantly reducing overhead.

Datasets

The authors evaluated HitGNN using four widely recognized large-scale graph datasets:

  • Reddit (RD): ~233k vertices, 23M edges.
  • Yelp (YP): ~717k vertices, 14M edges.
  • Amazon (AM): ~1.5M vertices, 264M edges.
  • ogbn-products (PR): ~2.4M vertices, 62M edges.

Key Insights and Ideas

  • Customization Trumps Raw Power: An Nvidia RTX A5000 GPU has vastly higher peak TFLOPS and memory bandwidth than a Xilinx Alveo U250 FPGA. However, because HitGNN optimizes the hardware specifically for the GNN's memory access patterns, it achieves much higher "effective" performance.
  • The Bottleneck Shift: Traditional intuition suggests that the "Aggregation" step (gathering neighbor data) is always the bottleneck. HitGNN’s DSE engine discovered that once communication is optimized, the bottleneck can shift to the "Update" step (the neural network part). The framework automatically reallocates hardware resources to the new bottleneck to maximize throughput.
  • Scalability via the Host: While direct chip-to-chip links (like NVLink for GPUs) are popular, HitGNN shows that for GNNs, using the CPU as a high-capacity data broker is surprisingly effective and much easier to implement on standard cloud FPGA platforms like AWS or Azure.

Key Results

  • Speedup: HitGNN achieved a geometric mean speedup of 2.11× to 2.34× over a 4-GPU baseline across different algorithms, with peak speedups reaching 4.26×.
  • Efficiency: The framework demonstrated up to 27.21× higher bandwidth efficiency. It essentially does more work with less "fuel" (memory bandwidth).
  • Optimization Impact: The combined workload balancing and communication optimizations improved throughput by up to 66%.
  • Scalability: The system scales almost linearly up to 16 FPGAs, proving it can handle the demands of massive, industrial-scale graphs.

Limitations and Future Work

The primary limitation identified is the CPU memory bandwidth. Since all FPGAs fetch missing data from the host CPU, the CPU's memory eventually becomes the bottleneck (estimated at around 12-13 FPGAs for their specific test setup).

To push past this, the authors suggest future work in:

  • Data Prefetching: Predicting which data an FPGA will need next to hide latency.
  • High-Speed Interconnects: Exploring the use of SmartNICs or direct FPGA-to-FPGA networking to offload the host CPU.

Summary updated

Abstract

As the size of real-world graphs increases, training Graph Neural Networks (GNNs) has become time-consuming and requires acceleration. While previous works have demonstrated the potential of utilizing FPGA for accelerating GNN training, few works have been carried out to accelerate GNN training with multiple FPGAs due to the necessity of hardware expertise and substantial development effort. To this end, we propose HitGNN, a framework that enables users to effortlessly map GNN training workloads onto a CPU+Multi-FPGA platform for acceleration. In particular, HitGNN takes the user-defined synchronous GNN training algorithm, GNN model, and platform metadata as input, determines the design parameters based on the platform metadata, and performs hardware mapping onto the CPU+Multi-FPGA platform, automatically. HitGNN consists of the following building blocks: (1) high-level application programming interfaces (APIs) that allow users to specify various synchronous GNN training algorithms and GNN models with only a handful of lines of code; (2) a software generator that generates a host program that performs mini-batch sampling, manages CPU-FPGA communication, and handles workload balancing among the FPGAs; (3) an accelerator generator that generates GNN kernels with optimized datapath and memory organization. We show that existing synchronous GNN training algorithms such as DistDGL and PaGraph can be easily deployed on a CPU+Multi-FPGA platform using our framework, while achieving high training throughput. Compared with the state-of-the-art frameworks that accelerate synchronous GNN training on a multi-GPU platform, HitGNN achieves up to 27.21× bandwidth efficiency, and up to 4.26× speedup using much less compute power and memory bandwidth than GPUs. In addition, HitGNN demonstrates good scalability to 16 FPGAs on a CPU+Multi-FPGA platform.
BibTeX
@article{lin2024hitgnn,
  author = {Yi-Chien Lin and Bingyi Zhang and Viktor K. Prasanna},
  title = {{HitGNN: High-Throughput GNN Training Framework on CPU+Multi-FPGA Heterogeneous Platform}},
  journal = {IEEE Trans. Parallel Distributed Syst.},
  year = {2024},
  url = {https://openreview.net/forum?id=vhDY3Aj9Bt},
}

Loading related content...

Figure