Abstract
Graph Neural Networks (GNNs) have emerged as powerful tools for graph-structured data, yet their deployment in real-time applications is hindered by high inference latency. This paper introduces a novel CPU-FPGA heterogeneous system designed to optimize mini-batch GNN inference. By integrating a decoupled GNN architecture with a flexible hardware accelerator featuring an Adaptive Computation Kernel (ACK), the system effectively addresses the "neighborhood explosion" and hardware load imbalance problems. Experimental results demonstrate speedups of up to 50x over CPUs and 21x over high-end GPUs, establishing a new state-of-the-art for low-latency GNN acceleration.
1. Introduction
Graph Neural Networks (GNNs) are increasingly critical for tasks such as recommendation systems, fraud detection, and social network analysis. However, deploying these models in production environments requires strict adherence to low-latency constraints. Traditional hardware architectures, including CPUs and GPUs, often struggle with the irregular memory access patterns and the computational diversity inherent in GNNs. This paper presents a co-designed hardware-software solution that leverages the strengths of both CPUs and FPGAs to achieve high-throughput, low-latency inference.
2. Challenges in GNN Acceleration
Two primary bottlenecks impede the performance of GNN inference:
2.1 Neighborhood Explosion
Standard GNNs utilize a recursive aggregation scheme where a node's representation is updated based on its $L$-hop neighbors. In many real-world graphs, this "receptive field" grows exponentially with the number of layers. This phenomenon, known as neighborhood explosion, results in massive data movement and computational overhead that quickly exceeds the memory bandwidth and capacity of standard hardware.
2.2 Hardware Load Imbalance
GNN layers typically consist of two distinct phases: Aggregation (sparse operations involving graph structure) and Transformation (dense operations involving feature matrix multiplications). Most existing hardware accelerators employ separate, dedicated modules for these tasks. Due to the irregular nature of graph data, these modules often experience significant load imbalance, where one module remains idle while the other is fully utilized, leading to suboptimal hardware efficiency.
3. Proposed Methodology
The authors propose a multi-level approach to overcome these challenges, spanning algorithmic decoupling and hardware innovation.
3.1 Model Depth-Receptive Field Decoupling
To mitigate neighborhood explosion, the system adopts "Decoupled GNNs." Instead of recursive neighbor searching, the system identifies a fixed set of "important neighbors" using Personalized PageRank (PPR). This creates a static, manageable subgraph for each target node. By decoupling the model depth from the receptive field, the computational complexity becomes linear relative to the number of layers, significantly reducing the data movement burden.
3.2 Adaptive Computation Kernel (ACK)
The core of the FPGA accelerator is the Adaptive Computation Kernel (ACK). Unlike rigid hybrid designs, the ACK is a unified processing element capable of reconfiguring its internal data paths:
- Systolic Mode: Optimized for high-speed dense matrix multiplications during the transformation phase.
- Scatter-Gather Mode: Optimized for sparse graph aggregations.
This flexibility ensures that hardware resources are fully utilized throughout the entire GNN layer execution, eliminating the idle time common in previous architectures.
3.3 Heterogeneous Task Scheduling
The system employs a CPU-FPGA pipeline where the CPU handles Important Neighbor Identification (INI) and subgraph construction, while the FPGA performs the heavy-duty inference. To maximize throughput, the authors implement double and triple buffering techniques to overlap data transfer with computation, effectively "hiding" communication latency.
4. Experimental Evaluation
The system was implemented on a Xilinx Alveo U250 board and evaluated against an AMD Ryzen 3990x CPU and an Nvidia RTX 3090 GPU.
4.1 Datasets and Models
The evaluation utilized three large-scale datasets:
- Flickr (FL): ~90k vertices, ~900k edges.
- Reddit (RE): ~233k vertices, ~116M edges.
- ogbn-arxiv (OA): ~169k vertices, ~1.1M edges.
The system was tested across three popular GNN variants: GCN, GraphSAGE, and GAT.
5. Key Results and Insights
The proposed heterogeneous platform demonstrates significant performance gains:
- Execution Speed: The system achieved 21.4x–50.8x speedup over CPU-only implementations and 2.9x–21.6x speedup over CPU-GPU configurations.
- Comparison with State-of-the-Art: The design outperformed GraphACT, a leading FPGA-based GNN accelerator, by a factor of 4.7x.
- Hardware Efficiency: The ACK design significantly improved hardware utilization. While previous hybrid designs suffered from up to 70% idle time, the ACK maintains high activity across both sparse and dense phases.
- Scalability: The system maintains millisecond-level latency even for deep 16-layer models, making it suitable for real-time industrial applications.
6. Conclusion and Future Work
This work demonstrates that the combination of algorithmic decoupling and adaptive hardware kernels can overcome the fundamental bottlenecks of GNN inference. By transforming a memory-bound problem into a compute-bound one and ensuring high hardware utilization, the proposed system provides a viable path for deploying deep GNNs in latency-sensitive environments.
Future research directions include addressing potential CPU bottlenecks in neighbor identification for extremely large batches and extending the architecture to support the complexities of GNN training and backpropagation.
Summary updated