Generalized Taxonomy-Guided Graph Neural Networks

Zhou Yu, Di Jin, Jianguo Wei

Introduction

In many real-world domains, such as citation networks, patent databases, and biological systems, data is naturally organized into hierarchical taxonomies. For instance, a research paper might be categorized under Computer Science → Artificial Intelligence → Computer Vision. While standard Graph Neural Networks (GNNs) excel at capturing local neighborhood structures, they often struggle to integrate the rich semantic and directional information inherent in these hierarchies. Existing methods frequently treat taxonomy labels as flat categories or fail to account for the specific roles (parent vs. child) within the hierarchy.

Furthermore, real-world taxonomies are often incomplete. In datasets like Yelp or PubMed, many nodes may be missing category tags or belong to sparse branches of the taxonomy. This paper introduces TG-GNN (Taxonomy-Guided Graph Neural Networks), a framework designed to learn generalized taxonomy representations that remain robust even when structural data is sparse or nodes are entirely new.

Methodology

The TG-GNN architecture consists of two synergistic modules designed to bridge the gap between raw graph data and structured taxonomic knowledge.

1. Taxonomy Representation Learning

To capture the "shape" of the hierarchy, the model constructs an ego network for every category. Unlike traditional models that treat categories as isolated points, TG-GNN identifies the role of each node in the ego network as a parent, anchor, or child. A Position-Enhanced GNN processes these roles alongside semantic features (initialized via Word2Vec). This allows the model to learn that a parent category relates differently to an anchor than a sibling does, capturing the directional flow of information.

2. Network Representation Learning

Once the taxonomy is mapped into a continuous vector space, it is used to guide the learning of the actual data network. The model employs a Graph Attention Network (GAT) to generate initial node embeddings. These embeddings are then refined through a Taxonomy-Guided Markov Mechanism. By utilizing a Markov Random Field (MRF), the model calculates a "pairwise potential" based on the similarity of nodes' paths within the taxonomy. This mechanism effectively regularizes the network, encouraging nodes with similar taxonomic backgrounds to share similar representations.

The TG-GNN architecture, illustrating the dual-module approach for taxonomy and network representation learning.

Experimental Evaluation

The authors evaluated TG-GNN across four diverse datasets: Aminer (co-authorship), Patent (citations), PubMed (proteins), and Yelp (businesses). Notably, PubMed and Yelp featured significantly incomplete taxonomies (20% and 66% coverage, respectively).

Key Results

  • Superior Performance: TG-GNN consistently outperformed state-of-the-art baselines, including GCN, GAT, and specialized models like TaxoGNN.
  • Robustness to Sparse Data: On the Yelp dataset, TG-GNN achieved a 21.07% improvement in accuracy over the best baseline, demonstrating that its structural learning approach can infer relationships even for untagged nodes.
  • Inductive Capability: In "cold start" scenarios where the model was tested on unseen parts of the graph, TG-GNN maintained high accuracy, proving it learns generalized principles rather than just memorizing the training set.

Interpretability and Visualization

One of the most compelling aspects of TG-GNN is its ability to organize the semantic space more logically than traditional methods. Visualization via t-SNE reveals that while Word2Vec often clusters categories based on linguistic naming conventions, TG-GNN produces tight, distinct clusters that mirror the actual hierarchical structure of the taxonomy.

Visualization of research topic embeddings: TG-GNN (c) produces much clearer, taxonomically-aligned clusters compared to Word2Vec (b).

Conclusion

TG-GNN represents a significant step forward in integrating external knowledge hierarchies into graph learning. By treating taxonomies as structured ego networks rather than flat labels, and using an MRF-based bridge to regularize node embeddings, the framework achieves state-of-the-art results and remarkable robustness to incomplete data. Future work may focus on adapting this framework for dynamic taxonomies that evolve over time and optimizing the computational efficiency of the MRF layers for billion-scale graphs.

Summary updated

BibTeX
@inproceedings{yu2024generalized,
  author = {Zhou Yu and Di Jin and Jianguo Wei},
  title = {{Generalized Taxonomy-Guided Graph Neural Networks}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2024},
  url = {https://openreview.net/forum?id=V6GSN9KsRH},
}

Loading related content...

Figure