An efficient high throughput system of hypergraph neural network under CPU-GPU heterogeneous environments

Hui Yu, Yu Zhang, Xintao Li, Zikang Chen, Ying‐Qi Zhao, Jin Zhao, Hao Qi, Xiaofei Liao, Hai JinScientia 2025 · DOI

Abstract

In recent years, graph neural networks (GNNs) have gained significant attention for their exceptional ability to learn and reason with non-Euclidean data. However, real-world graph structures often involve complex higher-order relationships between vertices, typically represented as hypergraphs. To effectively capture these intricate features and underlying semantic information in hypergraphs, numerous hypergraph neural network (HGNN) models have been proposed. Despite the development of various software systems that leverage the high parallel computing power of GPUs to accelerate HGNN training, these systems still face challenges such as excessive redundant computation and frequent data communication leading to suboptimal GPU utilization. Moreover, these systems are limited to training HGNN models on relatively small-scale graphs. To address these issues, we observed that multiple hyperedge computations often repeatedly access and compute the identical vertex feature vectors due to significant topological overlap between hyperedges in hypergraphs when handling the HyperGNN models. Based on this observation, we propose RHGNN, a redundancy-elimination HyperGNN training system tailored for CPU-GPU heterogeneous environments. RHGNN exploits the overlapping characteristics of hyperedges to optimize the computation and updating of hyperedge and vertex features, significantly reducing CPU-GPU data communication overhead. Specifically, RHGNN presents a novel hyperedge-centric redundancy elimination approach, enabling the system to load the feature vectors of vertices to optimize communication, allowing a single load operation to serve multiple hyperedge and vertex feature computations. In addition, RHGNN implements an efficient hyperdegree-aware hierarchy caching mechanism that prioritizes frequently accessed vertex feature vectors and intermediate results on the GPU, further reducing communication costs. To validate the effectiveness of RHGNN, we compared its performance against the state-of-the-art HGNN software systems DGL, PyG, and HyperGef. Experimental results demonstrate that RHGNN achieves a performance improvement of 2.5×∼3.4× over PyG, 2.1×∼3.1× over DGL, and 1.4×∼2.3× over HyperGef in HGNN model training, respectively.

Introduction: The Challenge of Hypergraph Learning

Standard Graph Neural Networks (GNNs) excel at modeling binary relationships (e.g., A is friends with B). However, many real-world scenarios involve complex, multi-way interactions, such as a research paper with multiple authors or a metabolic pathway involving a dozen proteins. These are best represented as hypergraphs, where a single "hyperedge" can connect any number of vertices.

The general workflow of a Hypergraph Neural Network (HGNN), illustrating the stages of feature projection, hyperedge aggregation, vertex aggregation, and vertex update.

While Hypergraph Neural Networks (HGNNs) effectively capture these higher-order relationships, their training is computationally expensive. Current frameworks (like PyG, DGL, and HyperGef) treat hyperedges somewhat independently, leading to two major bottlenecks:

  1. Redundant Computation: Because hyperedges in real-world data often overlap significantly (sharing many of the same vertices), systems end up fetching and calculating the same vertex features repeatedly. Analysis shows that in some datasets, over 82% of vertex and hyperedge computations are redundant.
  2. The Communication Wall: Frequent data movement between the CPU (where the graph is stored) and the GPU (where the math happens) creates a massive bottleneck, leaving the GPU idling while waiting for data.

The RHGNN System

To address these inefficiencies, we introduce RHGNN (Redundancy-elimination Hypergraph Neural Network), which tackles these issues through three core innovations:

1. Hyperedge-centric Redundancy Elimination

Instead of processing hyperedges in isolation, RHGNN groups "similar" hyperedges (those sharing many vertices) into the same execution block. It uses a Topological Similarity (TS) metric and a dynamic thresholding algorithm to partition the hypergraph. This allows the system to load a vertex's features once and reuse them for multiple hyperedge calculations, effectively implementing a "load once, compute many" strategy.

2. Hyperdegree-aware Hierarchical Caching

Real-world hypergraphs follow a power-law distribution—a few "celebrity" vertices appear in a huge number of hyperedges. RHGNN analysis found that the top 0.5% of highest-degree vertices account for over 73% of all data accesses. RHGNN implements a two-level cache to exploit this:

  • Level 1: Caches intermediate results for shared vertices within a specific hyperedge block.
  • Level 2: Identifies and permanently stores high-degree vertex features in the GPU's global memory to avoid repeated transfers.

3. Asynchronous Pipeline & Dual-Pointer Optimization

To maximize hardware utilization, RHGNN uses a dual-pointer iteration strategy to accelerate similarity calculations. It also implements an asynchronous scheduler that overlaps data loading with GPU computation, ensuring the hardware is rarely idle.

Experimental Evaluation

RHGNN was evaluated on five diverse, real-world hypergraph datasets, including Cooking200 (CK), CoauthorshipDBLP (DB), and WalmartTrips (WM). The system significantly outperforms existing industry-standard tools across all tested models (HGNN, HGCN, UniGNN):

  • Speedup: RHGNN is 2.5×–3.4× faster than PyG, 2.1×–3.1× faster than DGL, and 1.4×–2.3× faster than HyperGef.
  • Communication Efficiency: It reduces the volume of CPU-to-GPU data transfer by 47.2%–62.3% compared to PyG.
  • GPU Utilization: While other systems saw GPU utilization fluctuate due to data-loading stalls, RHGNN maintained a steady and high utilization rate, averaging ~93% on the WM dataset.

Conclusion

RHGNN provides a robust solution for high-throughput HGNN training by exploiting structural redundancies and optimizing data movement in CPU-GPU heterogeneous environments. Future work will focus on extending these redundancy-elimination strategies to distributed multi-node systems and dynamic hypergraphs that change in real-time.

Summary updated

Abstract

In recent years, graph neural networksGNNs) have gained significant attention for their exceptional ability to learn and reason with non-Euclidean data. However, real-world graph structures often involve complex higher-order relationships between vertices, typically represented as hypergraphs. To effectively capture these intricate features and underlying semantic information in hypergraphs, numerous hypergraph neural networkHGNN) models have been proposed. Despite the development of various software systems that leverage the high parallel computing power of GPUs to accelerate HGNN training, these systems still face challenges such as excessive redundant computationand frequent data communication leading to suboptimal GPU utilization. Moreover, these systems are limited to training HGNN models on relatively small-scale graphs. To address these issues, we observed that multiple hyperedge computations often repeatedly access and compute the identical vertex feature vectors due to significant topological overlap between hyperedges in hypergraphs when handling the HyperGNN models. Based on this observation, we propose RHGNN a redundancy-elimination HyperGNN training system tailored for CPU-GPU heterogeneous environments. RHGNN exploits the overlapping characteristics of hyperedges to optimize the computation and updating of hyperedge and vertex features, significantly reducing CPU-GPU data communication overhead. Specifically, RHGNN presents a novel hyperedge-centric redundancy elimination approach, enabling the system to load the feature vectors of vertices to optimize communication, allowing a single load operation to serve multiple hyperedge and vertex feature computations. In addition, RHGNN implements an efficient hyperdegree-aware hierarchy caching mechanism that prioritizes frequently accessed vertex feature vectors and intermediate results on the GPU, further reducing communication costs. To validate the effectiveness of RHGNN, we compared its performance against the state-of-the-art HGNN software systems DGL, PyG, and HyperGef. Experimental results demonstrate that RHGNN achieves a performance improvement of 2.5×∼3.4× over PyG, 2.1×∼3.1× over DGL, and 1.4×∼2.3× over HyperGef in HGNN model training, respectively.
BibTeX
@inproceedings{yu2025efficient,
  author = {Hui Yu and Yu Zhang and Xintao Li and Zikang Chen and Ying‐Qi Zhao and Jin Zhao and Hao Qi and Xiaofei Liao and Hai Jin},
  title = {{An efficient high throughput system of hypergraph neural network under CPU-GPU heterogeneous environments}},
  booktitle = {Scientia},
  year = {2025},
  doi = {10.1360/ssi-2024-0282},
}
Figure