Graph Topology Information Enhanced Heterogeneous Graph Representation Learning

He Zhao, Zhiwei Zeng, Yongwei Wang, Chunyan MiaoarXiv 2026 · arXiv

TL;DR

ToGRL is a novel framework that tackles the inherent noise and high memory overhead of learning from heterogeneous graphs. By decoupling graph structure learning into a memory-efficient two-stage process and introducing NLP-inspired soft prompt tuning, ToGRL extracts task-optimized topologies and significantly outperforms state-of-the-art models on downstream tasks.

What Problem Does This Paper Address?

Graph Neural Networks (GNNs) are highly sensitive to the quality of their input graph structures. In the real world, heterogeneous graphs—which contain multiple types of nodes and edges (e.g., users, products, and brands in an e-commerce network)—are inherently noisy and rarely optimized for downstream tasks.

This issue is amplified in heterogeneous graphs due to the "hub effect" of meta-paths: a single noisy node can generate multiple pairs of noisy meta-path-based neighbors, propagating errors across the network.

Figure 2: Noise effect is enlarged by changing from two-hop relationships (in homogeneous models on the right) to one- hop relationships(in heterogeneous models on the left).

While Graph Structure Learning (GSL) methods exist to clean up and optimize graph topologies, they are mostly designed for homogeneous graphs. Directly applying them to heterogeneous graphs leads to severe memory consumption issues because they typically try to optimize the entire $N \times N$ adjacency matrix simultaneously.

Methods

The authors propose ToGRL (Graph Topology learning Enhanced Heterogeneous Graph Representation Learning), which breaks the problem down into a memory-efficient pipeline:

  1. Mining Latent Graph Topology (Stage 1): Instead of optimizing a massive adjacency matrix directly, ToGRL uses a task-constrained random walk strategy (based on Node2Vec) to learn topology embeddings. It forces nodes within the same sampled walk to share downstream task labels, ensuring the topology captures task-relevant features.
  2. Constructing a Smooth Graph: Using principles from Graph Signal Processing (GSP), ToGRL treats the topology embeddings as node signals and constructs a new, sparse graph by keeping only the top-$k$ closest neighbors based on Euclidean distance. This ensures the new graph structure is "smooth" (i.e., connected nodes have similar embeddings).
Figure 3: An illustration of smoothness on graphs. The ar- rows represent node signals. The two graphs has the same graph signals while having different graph structures. The left one is smoother than the right one. In ToGRL, we treat the topology embeddings as the node signals and aim to con- struct a smooth graph.
  1. Graph Representation Learning (Stage 2): Final node representations are learned on this newly constructed graph using a GCN backbone combined with collaborative knowledge distillation to capture both regional and global context.
  2. Prompt Decoder: To bridge the gap between representation learning and downstream tasks (especially when labels are scarce), ToGRL introduces a prefix soft prompt tuning mechanism. A task-specific trainable vector is prepended to the learned embeddings, helping the decoder better utilize the learned knowledge without altering the underlying representations.
Figure 1: An overview of our ToGRL framework. First, a graph structure learning module is used to mine latent graph topology together with downstream task information. The learned topology embeddings are utilized to construct a new graph. Then, a representation learning module is used to learn final node representations. Finally, a prompt tuning decoder is used to evaluate embeddings on downstream tasks.

Datasets

The framework was evaluated on five standard real-world heterogeneous graph datasets:

  • ACM, DBLP, ACM2: Academic citation networks mapping relationships between authors, papers, conferences, and subjects.
  • IMDB: A movie network containing movies, actors, and directors.
  • PubMed: A medical network mapping genes, diseases, chemicals, and species.
Table 1: Summary of datasets statistics used in experi- ments.ACM, IMDB, DBLP and ACM2 are used for node clas- sification tasks. ACM2 and PubMed are used for the link prediction task.

Key Insights and Ideas

  • Decoupling is Key for Scalability: By separating the optimization of the graph structure from the node representation learning, ToGRL bypasses the $O(N^2)$ memory bottleneck that plagues traditional GSL methods.
  • Euclidean Distance Beats Inner Product for Smoothness: The authors found that using Euclidean distance to construct the new graph aligns much better with the mathematical definition of graph signal smoothness than standard inner-product distance, leading to cleaner graph structures.
  • Prompts Work for Graphs Too: Borrowing soft prompt tuning from NLP helps mitigate the "gap" between self-supervised graph representation learning and semi-supervised downstream tasks, proving highly effective when training data is limited.

Key Results

ToGRL was tested on node classification and link prediction tasks, consistently outperforming strong baselines (including SeHGNN, CKD, and GTN):

  • Performance Boost: On the highly noisy IMDB dataset, ToGRL outperformed the best baseline by 6.88% on Macro F1 and 5.51% on Micro F1 for node classification.
  • Memory Efficiency: While traditional GSL methods like HGSL suffered out-of-memory errors on larger datasets (like DBLP and ACM2) or consumed massive amounts of VRAM (11.7 GB on IMDB), ToGRL's structure learning module only required 2.7 GB on IMDB, matching the footprint of standard non-GSL models.

Limitations and Future Work

The authors note that their random walk strategy currently relies on a relatively simple Node2Vec implementation. They suggest that extending the framework to incorporate more advanced, heterogeneous-specific walking strategies (such as Metapath2Vec) could capture even richer semantic relationships and further boost performance.

Summary updated

Abstract

Real-world heterogeneous graphs are inherently noisy and usually not in the optimal graph structures for downstream tasks, which often adversely affects the performance of GRL models in downstream tasks. Although Graph Structure Learning (GSL) methods have been proposed to learn graph structures and downstream tasks simultaneously, existing methods are predominantly designed for homogeneous graphs, while GSL for heterogeneous graphs remains largely unexplored. Two challenges arise in this context. Firstly, the quality of the input graph structure has a more profound impact on GNN-based heterogeneous GRL models compared to their homogeneous counterparts. Secondly, most existing homogenous GRL models encounter memory consumption issues when applied directly to heterogeneous graphs. In this paper, we propose a novel Graph Topology learning Enhanced Heterogeneous Graph Representation Learning framework (ToGRL).ToGRL learns high-quality graph structures and representations for downstream tasks by incorporating task-relevant latent topology information. Specifically, a novel GSL module is first proposed to extract downstream task-related topology information from a raw graph structure and project it into topology embeddings. These embeddings are utilized to construct a new graph with smooth graph signals. This two-stage approach to GSL separates the optimization of the adjacency matrix from node representation learning to reduce memory consumption. Following this, a representation learning module takes the new graph as input to learn embeddings for downstream tasks. ToGRL also leverages prompt tuning to better utilize the knowledge embedded in learned representations, thus enhancing adaptability to downstream tasks. Extensive experiments on five real-world datasets show that our ToGRL outperforms state-of-the-art methods by a large margin.
BibTeX
@misc{zhao2026graph,
  author = {He Zhao and Zhiwei Zeng and Yongwei Wang and Chunyan Miao},
  title = {{Graph Topology Information Enhanced Heterogeneous Graph Representation Learning}},
  year = {2026},
  eprint = {2604.05732},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2604.05732},
}
Figure