Abstract
This paper presents GraphAGILE, a domain-specific FPGA-based overlay accelerator for graph neural network (GNN) inference. GraphAGILE consists of (1) a novel unified architecture design with an instruction set, and (2) a compiler built upon the instruction set that can quickly generate optimized code. Due to the proposed instruction set architecture (ISA) and the compiler, GraphAGILE does not require any FPGA reconfiguration when performing inference on various GNN models and input graphs. For the architecture design, we propose a novel hardware module named Adaptive Computation Kernel (ACK), that can execute various computation kernels of GNNs, including general matrix multiplication (GEMM), sparse-dense matrix multiplication (SpDMM) and sampled dense-dense matrix multiplication (SDDMM). The compiler takes the specifications of a GNN model and the graph meta data as input and generates a sequence of instructions for inference execution. We implement GraphAGILE on a state-of-the-art FPGA platform, Xilinx Alveo U250, achieving significant speedups over traditional CPU and GPU implementations.
Introduction
Graph Neural Networks (GNNs) are notoriously difficult to accelerate because they sit at the intersection of dense linear algebra and sparse graph processing. Current solutions face a "flexibility vs. performance" trade-off: general-purpose processors (CPUs/GPUs) struggle with irregular memory access patterns, while existing FPGA accelerators are often "hard-wired" for specific models. Changing a model or graph topology on traditional FPGAs typically requires hours of hardware re-synthesis, making them impractical for dynamic cloud environments. GraphAGILE addresses this by introducing an overlay approach—a "soft" processor built on the FPGA that is programmable via instructions, eliminating the need for hardware reconfiguration.
System Architecture and Compiler
The GraphAGILE framework consists of a specialized compiler and a hardware platform. The compiler transforms user-defined GNN models and input graphs into an instruction sequence based on a domain-specific 128-bit ISA. This allows the hardware to manage data movement and computation dynamically.
The compiler performs four critical optimizations to reduce latency:
- Computation Order Optimization: Mathematically determines the most efficient sequence of "Aggregation" and "Linear" layers to minimize operations.
- Layer Fusion: Merges adjacent layers to keep data on-chip, reducing external memory access.
- Partition-Centric Execution: Breaks large graphs into shards that fit perfectly into the FPGA's local memory.
- Task Scheduling: Uses double/triple buffering to overlap data transfers with computation.
Adaptive Computation Kernel (ACK)
At the heart of the hardware is the Adaptive Computation Kernel (ACK). This unified module reconfigures itself on-the-fly to execute three distinct types of math: General Matrix Multiplication (GEMM), Sparse-Dense Matrix Multiplication (SpDMM), and Sampled Dense-Dense Matrix Multiplication (SDDMM). By using a systolic array structure that can be repurposed for different kernels, the ACK maximizes hardware utilization across heterogeneous GNN operations.
Evaluation and Results
The system was evaluated using seven widely recognized graph datasets, ranging from small citation networks (Cora, Citeseer) to large-scale graphs like Amazon-Products (1.5M vertices, 264M edges). GraphAGILE successfully executed diverse models including GCN, GAT, GIN, GraphSAGE, and SGC.
Key performance metrics include:
- End-to-End Latency: GraphAGILE achieved up to a 47.1× reduction in latency compared to high-end CPUs and a 3.9× reduction compared to Nvidia RTX3090 GPUs.
- Hardware Efficiency: Compared to state-of-the-art FPGA accelerators like AWB-GCN, GraphAGILE showed up to a 2.9× speedup.
- Deployment Speed: By moving intelligence into the ISA and compiler, deployment time was reduced from hours (for synthesis) to milliseconds (for compilation).
Conclusion
GraphAGILE demonstrates that an overlay-based architecture can provide both high performance and software-like flexibility for GNN inference. While currently optimized for inference, future work includes extending the ISA to support training and multi-FPGA scaling for planetary-scale graphs. By eliminating the reconfiguration bottleneck, GraphAGILE paves the way for efficient GNN deployment in shared cloud environments.
Summary updated