HeMI: Multi-view Embedding in Heterogeneous Graphs

Costas Mavromatis, George KarypisarXiv 2021 · arXiv

Introduction

Real-world data is rarely uniform. From academic citation networks to movie databases, information is typically structured as Heterogeneous Graphs (HGs), where nodes and edges represent diverse entities and relationships. Learning effective representations (embeddings) for these graphs is crucial for downstream tasks like classification and recommendation. However, existing methods often fail to capture the rich, multi-faceted semantics inherent in HGs.

The Challenge of Heterogeneity

In an HG, a single node can be understood through multiple "views" defined by meta-paths—predefined sequences of node and edge types (e.g., Paper-Author-Paper). Traditional approaches often treat these views in isolation, creating information silos, or rely heavily on labeled data.

Figure 1: Illustration of node/edge types, a heterogeneous graph structure, and the concept of meta-paths.

The HeMI Framework: Learning through Consensus

To address these limitations, the authors propose HeMI (Heterogeneous graph Mutual Information maximization). HeMI is a self-supervised framework that treats different meta-paths as distinct views of the same graph and seeks a "consensus" representation.

Multi-view Mutual Information Maximization

The core innovation of HeMI lies in its objective function. Instead of simple spatial proximity, it maximizes the Mutual Information (MI) between a fused node embedding and two levels of graph structure:

  1. Fine-grain Consensus: Maximizing MI between the fused embedding ($\mathbf{z}_v$) and the specific representation of that node within a single meta-path view ($\mathbf{z}_v^{P_i}$).
  2. Coarse-grain Consensus: Maximizing MI between the fused embedding and a global summary of the entire meta-path graph ($\mathbf{s}^{P_i}$).
Figure 5: The HeMI architecture, showing how node views from different meta-paths are encoded, fused, and aligned through fine-grain and coarse-grain consensus.

By optimizing these objectives, HeMI forces the model to capture both local neighborhood features and global structural patterns that are consistent across different views.

Experimental Results

HeMI was evaluated on four benchmark datasets: ACM, DBLP, IMDB, and Last.fm.

  • Unsupervised Performance: HeMI consistently outperformed state-of-the-art unsupervised methods in node classification and clustering. Notably, on the IMDB dataset, HeMI achieved a significant jump in Normalized Mutual Information (NMI) compared to previous benchmarks.
  • Semi-supervised Boost: When used as an auxiliary loss for supervised models like MAGNN, HeMI improved link prediction performance by 8-10%, particularly in scenarios with limited labeled data.

Conclusion

HeMI demonstrates that maximizing mutual information across multiple structural views is a powerful paradigm for heterogeneous graph representation learning. By moving beyond simple neighborhood aggregation to a global consensus model, HeMI achieves high-quality embeddings without the need for extensive manual labeling. Future research may focus on automating meta-path discovery to further generalize the framework.

Summary updated

Abstract

Many real-world graphs involve different types of nodes and relations between nodes, being heterogeneous by nature. The representation learning of heterogeneous graphs (HGs) embeds the rich structure and semantics of such graphs into a low-dimensional space and facilitates various data mining tasks, such as node classification, node clustering, and link prediction. In this paper, we propose a self-supervised method that learns HG representations by relying on knowledge exchange and discovery among different HG structural semantics (meta-paths). Specifically, by maximizing the mutual information of meta-path representations, we promote meta-path information fusion and consensus, and ensure that globally shared semantics are encoded. By extensive experiments on node classification, node clustering, and link prediction tasks, we show that the proposed self-supervision both outperforms and improves competing methods by 1% and up to 10% for all tasks.
BibTeX
@misc{mavromatis2021hemi,
  author = {Costas Mavromatis and George Karypis},
  title = {{HeMI: Multi-view Embedding in Heterogeneous Graphs}},
  year = {2021},
  eprint = {2109.07008},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2109.07008},
}
Figure