Heterogeneous Network Representation Learning: A Unified Framework With Survey and Benchmark

Carl Yang, Yuxin Xiao, Yu Zhang, Yizhou Sun, Jiawei Han

TL;DR

This paper tackles the challenge of understanding and evaluating heterogeneous network representation learning (HNE) algorithms. The authors propose a unified framework for categorizing existing methods, introduce four novel benchmark datasets, and provide a comprehensive empirical comparison of 13 popular HNE algorithms, aiming to standardize research in this rapidly growing field.

What Problem Does This Paper Address?

Real-world data is inherently complex, involving diverse types of entities (nodes) and relationships (links). Heterogeneous networks are designed to model this complexity, offering a more realistic representation than traditional homogeneous graphs. Representation learning, or embedding, aims to capture the rich topological information within these networks by mapping nodes to low-dimensional vectors. However, the field of Heterogeneous Network Embedding (HNE) has become fragmented, with algorithms developed in disparate communities and evaluated on different datasets. This makes it difficult to compare methods, attribute performance gains to technical novelty versus data preprocessing, and hinders reproducible research. This paper aims to unify this landscape by providing a systematic survey, a standardized benchmark, and a comparative analysis of existing HNE algorithms.

Methods

The authors introduce a three-pronged approach:

  1. Unified Framework and Taxonomy: They propose a generic paradigm for HNE, based on network smoothness enforcement, which allows for the systematic categorization and analysis of existing and future algorithms. This framework helps understand the core merits of different models.
  2. Benchmark Datasets: To address the lack of standardized evaluation, they curate four novel heterogeneous network datasets from diverse domains (publication, recommendation, knowledge base, and biomedicine). These datasets vary in scale, structure, and attribute/label availability, offering a comprehensive testbed.
  3. Empirical Evaluation and Refactoring: They refactor and amend the implementations of 13 popular HNE algorithms, creating user-friendly interfaces for fair comparison. They then conduct extensive empirical evaluations on their benchmark datasets across various settings (unsupervised, attributed, and semi-supervised HNE) and tasks (node classification and link prediction).

Datasets

The paper introduces four new benchmark datasets:

  • DBLP: A network of authors, papers, venues, and phrases from DBLP.
  • Yelp: A network of businesses, users, locations, and reviews from Yelp.
  • Freebase: A network of books, films, music, sports, people, locations, organizations, and businesses from Freebase.
  • PubMed: A network of genes, diseases, chemicals, and species from PubMed.

These datasets are designed to cover a wide spectrum of application domains and possess diverse properties regarding scale, structure, attribute/label availability, and more.

Key Insights and Ideas

  • The fragmentation of HNE research is a significant bottleneck: The lack of standardized benchmarks and systematic comparisons makes it hard to assess true progress.
  • A unified framework is crucial for understanding and developing HNE: By categorizing algorithms based on their core objectives (proximity-preserving, message-passing, relation-learning), researchers can better grasp their strengths and weaknesses.
  • Data characteristics heavily influence algorithm performance: The paper highlights how properties like degree distribution, clustering coefficient, and meta-path distribution can bias HNE algorithms, emphasizing the need for diverse benchmarks.
  • Message-passing methods excel when attributes and labels are available: While proximity-preserving methods perform reasonably well in unsupervised settings, deep learning-based message-passing approaches like R-GCN, HAN, and MAGNN show significant gains when leveraging node attributes and labels.
  • Different tasks require different approaches: Node classification is more sensitive to node attributes and labels, while link prediction can sometimes be performed effectively even with limited or noisy data.

Key Results

  • The paper provides a comprehensive comparison of 13 HNE algorithms across four datasets and three settings (unsupervised, attributed, semi-supervised).
  • Proximity-preserving algorithms generally perform well in unsupervised settings for both node classification and link prediction.
  • Message-passing algorithms, particularly HGT, show competitive results even in unsupervised settings, while R-GCN, HAN, and MAGNN significantly outperform when attributes and labels are utilized.
  • Relation-learning methods like TransE and ComplEx perform better on datasets with more link types, such as Freebase and PubMed.
  • The study demonstrates that embedding size and link removal can significantly impact performance, highlighting the importance of robust evaluation protocols.
  • Adding random noise to node attributes can degrade node classification performance more than link prediction, suggesting that node content is more sensitive to noise for classification tasks.

Limitations and Future Work

  • Limitations: The paper acknowledges that while their framework is comprehensive, there might be other HNE algorithms that do not fit neatly into their taxonomy. The empirical evaluation is limited to 13 algorithms, and further research could explore a wider range.
  • Future Work: The authors suggest several promising directions:
  • Beyond Homophily: Exploring how to generalize principles from homogeneous networks (e.g., positional and structural embedding) to HNE.
  • Beyond Accuracy: Investigating scalability, efficiency, temporal adaptability, robustness to adversarial attacks, interpretability, uncertainty, and fairness of HNE algorithms.
  • Beyond Node Embedding: Extending HNE to graph- and subgraph-level embeddings for classification and unsupervised training.
  • Revisiting KB Embedding: Studying the intersection of KB embedding and HNE, particularly how to combine meta-path semantics with KB embedding techniques.
  • Modeling Heterogeneous Contexts: Developing methods to model rich content and contextual information associated with nodes, links, and subnetworks in heterogeneous networks.

Summary updated

Abstract

Since real-world objects and their interactions are often multi-modal and multi-typed, heterogeneous networks have been widely used as a more powerful, realistic, and generic superclass of traditional homogeneous networks (graphs). Meanwhile, representation learning ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">a.k.a.</i> embedding) has recently been intensively studied and shown effective for various network mining and analytical tasks. In this work, we aim to provide a unified framework to deeply summarize and evaluate existing research on heterogeneous network embedding (HNE), which includes but goes beyond a normal survey. Since there has already been a broad body of HNE algorithms, as the first contribution of this article, we provide a generic paradigm for the systematic categorization and analysis over the merits of various existing HNE algorithms. Moreover, existing HNE algorithms, though mostly claimed generic, are often evaluated on different datasets. Understandable due to the application favor of HNE, such indirect comparisons largely hinder the proper attribution of improved task performance towards effective data preprocessing and novel technical design, especially considering the various ways possible to construct a heterogeneous network from real-world application data. Therefore, as the second contribution, we create four benchmark datasets with various properties regarding scale, structure, attribute/label availability, and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">etc</i> . from different sources, towards handy and fair evaluations of HNE algorithms. As the third contribution, we carefully refactor and amend the implementations and create friendly interfaces for 13 popular HNE algorithms, and provide all-around comparisons among them over multiple tasks and experimental settings. By putting all existing HNE algorithms under a unified framework, we aim to provide a universal reference and guideline for the understanding and development of HNE algorithms. Meanwhile, by open-sourcing all data and code, we envision to serve the community with an ready-to-use benchmark platform to test and compare the performance of existing and future HNE algorithms ( <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/yangji9181/HNE</uri> ).
BibTeX
@article{yang2022heterogeneous,
  author = {Carl Yang and Yuxin Xiao and Yu Zhang and Yizhou Sun and Jiawei Han},
  title = {{Heterogeneous Network Representation Learning: A Unified Framework With Survey and Benchmark}},
  journal = {IEEE Trans. Knowl. Data Eng.},
  year = {2022},
  doi = {10.1109/tkde.2020.3045924},
}

Loading related content...

Figure