Abstract
Graph neural networks have been demonstrated to be effective analytic apparatus for mining network data. Most real-world networks are inherently hierarchical, offering unique opportunities to acquire latent, intrinsic network organizational properties by utilizing network taxonomies. The existing approaches for learning implicit hierarchical network structures focus on introducing taxonomy to graph neural networks but often run short of exploiting the rich network semantics and structural properties in the taxonomy, resulting in poor generalizability and reusability. To address these issues, we propose generalized Taxonomy-Guided Graph Neural Networks (TG-GNN) to integrate taxonomy into network representation learning. We first construct a taxonomy representation learning module that introduces the concept of ego network to propagate and aggregate rich semantic and structural information in the taxonomy. We then design a taxonomy-guided Markov mechanism, which encapsulates taxonomy knowledge in pairwise potential functions, to refine network embeddings. Extensive experiments on various real-world networks illustrate the effectiveness of TG-GNN over the state-of-the-art methods on scenarios involving incomplete taxonomies and inductive settings.
Introduction
In many domains, such as citation analysis or product recommendation, data is not merely a collection of flat connections. Instead, it is governed by an underlying hierarchy or taxonomy. For instance, a research paper on "Object Detection" is naturally categorized under "Computer Vision," which itself is a sub-field of "Artificial Intelligence."
While standard Graph Neural Networks (GNNs) are proficient at aggregating information from local neighborhoods, they often fail to capture these latent organizational properties. Existing methods that attempt to incorporate hierarchy often rely on inferred structures—which can be inaccurate—or use provided taxonomies too simplistically, ignoring the semantic depth of category names or the specific directional nature of parent-child relationships. This lack of nuance makes them fragile when faced with incomplete taxonomies or novel data.
The TG-GNN Framework
The proposed TG-GNN (Taxonomy-Guided Graph Neural Network) addresses these limitations through a dual-module architecture designed to explicitly encode hierarchical knowledge into the learning process.
1. Taxonomy Representation Learning
Rather than treating categories as static labels, this module views each category as an "anchor" within its own ego network, comprising the node itself, its parent, and its children.
- Semantic Encoding: Word2Vec is utilized to capture the linguistic meaning of category names.
- Position-Aware Embeddings: To distinguish between hierarchical roles, the model introduces embeddings that differentiate between parent, child, and anchor nodes.
- Position-Enhanced GNN: This component aggregates semantic and structural information, ensuring that a category's representation reflects its specific location and context within the broader taxonomy.
2. Network Representation Learning
This module applies the learned taxonomy embeddings to the actual data network (e.g., a citation or co-authorship graph).
- Initial Feature Extraction: A standard Graph Attention Network (GAT) is used to derive initial node features.
- Taxonomy-Guided Markov Mechanism: The core innovation is the use of a Markov Random Field (MRF) to refine node embeddings. By defining pairwise potential functions based on taxonomy similarity ($S^{taxo}$), the model encourages nodes with similar taxonomic paths to have closer representations, effectively "nudging" the network embeddings toward hierarchical consistency.
Experimental Evaluation
TG-GNN was evaluated across four diverse real-world datasets: Aminer (co-author network), Patent (citation dataset), PubMed (protein network), and Yelp (business network). Notably, the PubMed and Yelp datasets featured incomplete taxonomies (20% and 66% coverage, respectively) to test the model's robustness.
Key Results and Insights
- Performance Gains: TG-GNN consistently outperformed state-of-the-art baselines like GCN, GAT, and TaxoGNN. On the Yelp dataset, it achieved a 21.07% improvement in accuracy over the best baseline.
- Robustness to Incompleteness: The model proved remarkably resilient to missing taxonomic data. In the Yelp and PubMed datasets, the Markov mechanism successfully propagated hierarchical information from labeled to unlabeled nodes, resulting in F1 score gains of up to 20.88%.
- Generalization (Inductive Learning): In "cold start" scenarios—where the model is tested on previously unseen parts of the network—TG-GNN maintained its performance lead, demonstrating strong generalizability.
- Semantic-Structural Synergy: By combining the linguistic meaning of categories with their structural position, TG-GNN creates a more logical and nuanced mapping of the data space compared to models relying on semantics alone.
Conclusion and Future Work
TG-GNN demonstrates that explicitly modeling hierarchical taxonomies significantly enhances GNN performance and robustness. While the current framework assumes a fixed "ground truth" taxonomy, future research could explore dynamic taxonomy updates to account for evolving networks. Additionally, further optimization of the Markov refinement step may be necessary to scale the model to massive graphs containing millions of nodes.
Summary updated