TL;DR
ToGRL is a novel framework that tackles the inherent noise and high memory overhead of learning from heterogeneous graphs. By decoupling graph structure learning into a memory-efficient two-stage process and introducing NLP-inspired soft prompt tuning, ToGRL extracts task-optimized topologies and significantly outperforms state-of-the-art models on downstream tasks.
What Problem Does This Paper Address?
Graph Neural Networks (GNNs) are highly sensitive to the quality of their input graph structures. In the real world, heterogeneous graphs—which contain multiple types of nodes and edges (e.g., users, products, and brands in an e-commerce network)—are inherently noisy and rarely optimized for downstream tasks.
This issue is amplified in heterogeneous graphs due to the "hub effect" of meta-paths: a single noisy node can generate multiple pairs of noisy meta-path-based neighbors, propagating errors across the network.
While Graph Structure Learning (GSL) methods exist to clean up and optimize graph topologies, they are mostly designed for homogeneous graphs. Directly applying them to heterogeneous graphs leads to severe memory consumption issues because they typically try to optimize the entire $N \times N$ adjacency matrix simultaneously.
Methods
The authors propose ToGRL (Graph Topology learning Enhanced Heterogeneous Graph Representation Learning), which breaks the problem down into a memory-efficient pipeline:
- Mining Latent Graph Topology (Stage 1): Instead of optimizing a massive adjacency matrix directly, ToGRL uses a task-constrained random walk strategy (based on Node2Vec) to learn topology embeddings. It forces nodes within the same sampled walk to share downstream task labels, ensuring the topology captures task-relevant features.
- Constructing a Smooth Graph: Using principles from Graph Signal Processing (GSP), ToGRL treats the topology embeddings as node signals and constructs a new, sparse graph by keeping only the top-$k$ closest neighbors based on Euclidean distance. This ensures the new graph structure is "smooth" (i.e., connected nodes have similar embeddings).
- Graph Representation Learning (Stage 2): Final node representations are learned on this newly constructed graph using a GCN backbone combined with collaborative knowledge distillation to capture both regional and global context.
- Prompt Decoder: To bridge the gap between representation learning and downstream tasks (especially when labels are scarce), ToGRL introduces a prefix soft prompt tuning mechanism. A task-specific trainable vector is prepended to the learned embeddings, helping the decoder better utilize the learned knowledge without altering the underlying representations.
Datasets
The framework was evaluated on five standard real-world heterogeneous graph datasets:
- ACM, DBLP, ACM2: Academic citation networks mapping relationships between authors, papers, conferences, and subjects.
- IMDB: A movie network containing movies, actors, and directors.
- PubMed: A medical network mapping genes, diseases, chemicals, and species.
Key Insights and Ideas
- Decoupling is Key for Scalability: By separating the optimization of the graph structure from the node representation learning, ToGRL bypasses the $O(N^2)$ memory bottleneck that plagues traditional GSL methods.
- Euclidean Distance Beats Inner Product for Smoothness: The authors found that using Euclidean distance to construct the new graph aligns much better with the mathematical definition of graph signal smoothness than standard inner-product distance, leading to cleaner graph structures.
- Prompts Work for Graphs Too: Borrowing soft prompt tuning from NLP helps mitigate the "gap" between self-supervised graph representation learning and semi-supervised downstream tasks, proving highly effective when training data is limited.
Key Results
ToGRL was tested on node classification and link prediction tasks, consistently outperforming strong baselines (including SeHGNN, CKD, and GTN):
- Performance Boost: On the highly noisy IMDB dataset, ToGRL outperformed the best baseline by 6.88% on Macro F1 and 5.51% on Micro F1 for node classification.
- Memory Efficiency: While traditional GSL methods like HGSL suffered out-of-memory errors on larger datasets (like DBLP and ACM2) or consumed massive amounts of VRAM (11.7 GB on IMDB), ToGRL's structure learning module only required 2.7 GB on IMDB, matching the footprint of standard non-GSL models.
Limitations and Future Work
The authors note that their random walk strategy currently relies on a relatively simple Node2Vec implementation. They suggest that extending the framework to incorporate more advanced, heterogeneous-specific walking strategies (such as Metapath2Vec) could capture even richer semantic relationships and further boost performance.
Summary updated