Dual Hierarchical Attention Networks for Bi-typed Heterogeneous Graph Learning

Yu Zhao, Shaopeng Wei, Huaming Du, Xingyan Chen, Qing Li, Fuzhen Zhuang, Ji Liu, Gang Kou

Introduction

Graph Neural Networks (GNNs) have become a cornerstone for modeling complex relational data. While heterogeneous graphs—those containing multiple node and edge types—are common, a significant subset is the Bi-typed Multi-relational Heterogeneous Graph (BMHG). These graphs, such as academic networks (authors and papers) or e-commerce platforms (users and products), present unique structural characteristics. In a BMHG, relationships can be categorized into intra-class (within the same node type) and inter-class (between different node types).

Example of a Bi-typed Multi-relational Heterogeneous Graph (BMHG) with Author and Paper nodes.

The Structural Challenge of Bi-typed Graphs

Existing heterogeneous GNNs often overlook the fundamental differences between intra-class and inter-class relations. For instance, in an academic network, a "citation" link between papers carries different semantic weight than an "authorship" link between an author and a paper. Standard models that aggregate information uniformly across all relation types suffer from structural information loss. Furthermore, the importance of a relationship can be viewed from both a global perspective (general relevance of a relation type) and a local perspective (relevance to a specific node). Failing to capture this duality limits the model's representational power.

The DHAN Architecture

To address these challenges, the Dual Hierarchical Attention Network (DHAN) architecture is proposed. DHAN is specifically designed to exploit the bi-typed nature of BMHGs through two primary innovations: dual encoders and hierarchical attention.

The DHAN architecture featuring dual encoders and hierarchical attention mechanisms.

The model employs an Intra-class Encoder to process neighbors of the same type and an Inter-class Encoder to handle interactions between different types. This separation ensures that the distinct semantic signals from different relationship categories are preserved. Within these encoders, a hierarchical attention mechanism operates at two levels:

  1. Node-level Attention: Identifies the most influential neighbors for a specific target node.
  2. Relation-level Attention: Determines the relative importance of different relationship types (e.g., "same venue" vs. "same field").

A key feature of DHAN is its Global-Local mechanism. It combines a global weight for each relation type with a node-specific local weight, enhancing robustness against noise and preventing the model from converging to local optima.

Performance and Insights

DHAN was evaluated on three datasets derived from the Open Academic Graph (OAG): OAG1Y, OAG2Y, and OAG10Y. These datasets consist of author and paper nodes with various intra-class (colleague, cite, same venue, same field) and inter-class (important author, ordinary author) relations.

The results demonstrate DHAN's superiority over state-of-the-art models like GAT, HAN, and HGT. In the task of Author Disambiguation (AD), DHAN achieved an accuracy of 0.6394 on the OAG1Y dataset, significantly outperforming the next best model, ie-HGCN (0.5106). In node clustering tasks, DHAN showed improvements of up to 60% in Adjusted Rand Index. Visualization using t-SNE further confirmed that DHAN learns cleaner, more distinct representations compared to traditional methods.

Conclusion

DHAN provides a specialized and effective framework for learning from bi-typed heterogeneous graphs. By distinguishing between intra-class and inter-class signals and employing a global-local hierarchical attention mechanism, it captures the nuanced structural information inherent in these networks. Future research will focus on generalizing this architecture to multi-typed graphs ($|T| > 2$) and adapting it for dynamic graph environments where relationships evolve over time.

Summary updated

Abstract

Bi-typed multi-relational heterogeneous graph (BMHG) is one of the most common graphs in practice, for example, academic networks, e-commerce user behavior graph and enterprise knowledge graph. It is a critical and challenge problem on how to learn the numerical representation for each node to characterize subtle structures. However, most previous studies treat all node relations in BMHG as the same class of relation without distinguishing the different characteristics between the intra-type relations and inter-type relations of the bi-typed nodes, causing the loss of significant structure information. To address this issue, we propose a novel <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">D</b> ual <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">H</b> ierarchical <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">A</b> ttention <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</b> etworks (DHAN) based on the bi-typed multi-relational heterogeneous graphs to learn comprehensive node representations with the intra-type and inter-type attention-based encoder under a hierarchical mechanism. Specifically, the former encoder aggregates information from the same type of nodes, while the latter aggregates node representations from its different types of neighbors. Moreover, to sufficiently model node multi-relational information in BMHG, we adopt a newly proposed hierarchical mechanism. By doing so, the proposed dual hierarchical attention operations enable our model to fully capture the complex structures of the bi-typed multi-relational heterogeneous graphs. Experimental results on various tasks against the state-of-the-arts sufficiently confirm the capability of DHAN in learning node representations on the BMHGs.
BibTeX
@article{zhao2023dual,
  author = {Yu Zhao and Shaopeng Wei and Huaming Du and Xingyan Chen and Qing Li and Fuzhen Zhuang and Ji Liu and Gang Kou},
  title = {{Dual Hierarchical Attention Networks for Bi-typed Heterogeneous Graph Learning}},
  journal = {IEEE Trans. Knowl. Data Eng.},
  year = {2023},
  doi = {10.1109/tkde.2022.3221099},
}
Figure