Bottleneck Analysis of Dynamic Graph Neural Network Inference on CPU and GPU

Hanqiu Chen, Yahya Alhinai, Yihan Jiang, Eunjee Na, Cong HaoarXiv 2022 · OpenReview

TL;DR

This paper dives deep into the hardware performance of Dynamic Graph Neural Networks (DGNNs) on CPUs and GPUs, identifying key bottlenecks like temporal data dependencies and data movement. The authors provide a comprehensive analysis and propose optimizations to bridge the gap between algorithmic success and efficient hardware deployment.

What Problem Does This Paper Address?

Dynamic Graph Neural Networks (DGNNs) are powerful tools for analyzing data that changes over time, like social networks or traffic patterns. While algorithms for DGNNs have advanced significantly, deploying them on real-world hardware (CPUs and GPUs) presents a major challenge. Unlike static graphs, DGNNs must handle evolving structures and temporal information, making existing hardware optimizations for static graphs insufficient. This paper aims to bridge this gap by systematically analyzing the performance bottlenecks of popular DGNN models on both CPUs and GPUs, offering insights for future hardware-aware DGNN development.

Methods

The researchers selected eight prominent DGNN models with diverse characteristics and profiled their inference performance on both CPU and GPU. They utilized PyTorch Profiler to analyze memory usage and execution times of different model components and NVIDIA Nsight Systems for a detailed breakdown of CUDA and kernel operations. This profiling allowed them to identify and categorize common hardware bottlenecks. Based on these findings, they propose a set of optimizations spanning software, hardware, and co-design approaches.

Datasets

The paper mentions several datasets used for profiling, including:

  • Bitcoin Alpha
  • Reddit Hyperlink Network
  • Stochastic Block Model
  • Wikipedia
  • Caltrans Performance Measurement System (PeMS)
  • Social Evolution
  • GitHub-data
  • LastFM
  • Molecular Trajectories ISO17 Dataset

Key Insights and Ideas

  • Temporal Data Dependency is a Major Hurdle: The inherent sequential nature of processing time-series data in DGNNs, where each step depends on the previous one, severely limits parallelization on hardware, leading to underutilized GPU resources.
  • Data Movement is Costly: Frequent transfers of graph topology and node embeddings between CPU and GPU are a significant bottleneck, especially for dynamic graphs where this information changes constantly.
  • Workload Imbalance Between CPU and GPU: Graph sampling, often performed on the CPU, can become a bottleneck, leaving the GPU idle while waiting for data. Conversely, large batch sizes can saturate the GPU, leaving the CPU idle.
  • GPU Warm-up Overhead is Non-Trivial: Initializing the GPU, loading model weights, and setting up CUDA contexts introduce a significant overhead, particularly for smaller inference tasks.
  • Existing Static GNN Optimizations Don't Directly Translate: The unique temporal dynamics of DGNNs require new approaches beyond those developed for static graph processing.

Key Results

The paper provides a detailed breakdown of performance bottlenecks for eight DGNN models. While specific quantitative speedup numbers are not the primary focus, the analysis highlights significant GPU underutilization across various models, often below 1% for some components. For instance, TGAT showed low GPU utilization (5-6%) due to its sequential processing of nodes within a mini-batch. JODIE also exhibited very low GPU utilization (1.5-2.5%) despite using t-batching. The paper illustrates these bottlenecks with detailed profiling results and execution time breakdowns, showing how different modules contribute to the overall inference time.

Limitations and Future Work

The authors acknowledge that this work focuses solely on inference bottlenecks and does not cover the more complex training phase of DGNNs. They suggest that future research should investigate training bottlenecks. Additionally, they propose several optimization strategies but note that these have not yet been experimentally verified. The paper's future work includes profiling DGNN training and designing a dedicated DGNN hardware accelerator to implement the proposed optimizations and push the boundaries of efficient DGNN hardware design.

Summary updated

Abstract

Dynamic graph neural network (DGNN) is becoming increasingly popular because of its widespread use in capturing dynamic features in the real world. A variety of dynamic graph neural networks designed from algorithmic perspectives have succeeded in incorporating temporal information into graph processing. Despite the promising algorithmic performance, deploying DGNNs on hardware presents additional challenges due to the model complexity, diversity, and the nature of the time dependency. Meanwhile, the differences between DGNNs and static graph neural networks make hardware-related optimizations for static graph neural networks unsuitable for DGNNs. In this paper, we select eight prevailing DGNNs with different characteristics and profile them on both CPU and GPU. The profiling results are summarized and analyzed, providing in-depth insights into the bottlenecks of DGNNs on hardware and identifying potential optimization opportunities for future DGNN acceleration. Followed by a comprehensive survey, we provide a detailed analysis of DGNN performance bottlenecks on hardware, including temporal data dependency, workload imbalance, data movement, and GPU warm-up. We suggest several optimizations from both software and hardware perspectives. This paper is the first to provide an in-depth analysis of the hardware performance of DGNN Code is available at https://github.com/sharc-lab/DGNN_analysis.
BibTeX
@misc{chen2022bottleneck,
  author = {Hanqiu Chen and Yahya Alhinai and Yihan Jiang and Eunjee Na and Cong Hao},
  title = {{Bottleneck Analysis of Dynamic Graph Neural Network Inference on CPU and GPU}},
  year = {2022},
  eprint = {2210.03900},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2210.03900},
}
Figure