Benchmarking GPU and TPU Performance with Graph Neural Networks

Xiangyang Ju, Yunsong Wang, Daniel Murnane, Nicholas Choma, Steven Farrell, Paolo CalafiuraarXiv 2022 · DOI · arXiv

Introduction

AI accelerators like GPUs and TPUs are optimized for dense matrix operations typical of CNNs and Transformers. However, scientific domains such as High Energy Physics (HEP) often rely on sparse data structures represented as graphs. This study benchmarks NVIDIA GPUs and Google TPUs using a Graph Neural Network (GNN) designed for particle track reconstruction at the Large Hadron Collider (LHC), evaluating their performance on irregular workloads.

Methodology

The researchers employed an Interaction Network architecture, comprising an encoder, a relational "processor," and a decoder. A primary challenge in GNN training on accelerators is the dynamic shape problem. Unlike GPUs, TPUs require fixed input sizes for XLA compilation. To mitigate the overhead of padding every graph to a global maximum, the authors implemented a strategy of padding to the 99th percentile of graph sizes, resulting in a 30% reduction in training time. Performance was further evaluated using Roofline Analysis to identify whether the model was compute-bound or memory-bound.

Dataset and Task

The benchmark utilized the TrackML dataset, which simulates proton-proton collisions. Each event is represented as a graph with approximately 50,000 nodes (detector hits) and 250,000 edges (potential paths). The objective is a binary classification task: identifying which edges correspond to legitimate particle tracks.

Key Insights

Profiling revealed a significant bottleneck in "message passing," specifically the UnsortedSegmentSum operation in TensorFlow. TPUs spent 37.7% of execution time on this operation. Furthermore, while TPUs offer high theoretical peak TFLOPS, the GNN achieved only 2.3% utilization on TPU v3, compared to ~30% on the NVIDIA V100. This discrepancy underscores that peak performance metrics often fail to reflect efficiency for sparse data.

Key Results

Accuracy and Model Quality

Both hardware platforms achieved nearly identical performance, with Area Under the Curve (AUC) scores approximately 0.99. This confirms that the choice of accelerator does not impact the final model accuracy.

ROC curves for TPU and GPU models showing high classification accuracy.

Latency and Throughput

In terms of raw speed, a single 8-core TPU v3 performed comparably to two V100 GPUs, while a 32-core TPU v2 matched the throughput of four V100s.

Cost and Energy Efficiency

Despite the TPU's throughput, the GPU proved more efficient. Based on Google Cloud Platform pricing, GPUs offered a lower cost per training epoch. Additionally, the GPU demonstrated lower total energy consumption, making it a more sustainable choice for this specific workload.

Comparison of training cost per epoch across different hardware configurations.

Conclusion

While TPUs are powerful for dense workloads, GPUs currently provide better flexibility and cost-efficiency for sparse GNN applications. Future work will explore the impact of NVIDIA’s Tensor Cores and specialized hardware like Graphcore IPUs, which are designed specifically for irregular memory access patterns.

Summary updated

BibTeX
@misc{ju2022benchmarking,
  author = {Xiangyang Ju and Yunsong Wang and Daniel Murnane and Nicholas Choma and Steven Farrell and Paolo Calafiura},
  title = {{Benchmarking GPU and TPU Performance with Graph Neural Networks}},
  year = {2022},
  eprint = {2210.12247},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2210.12247},
}
Figure