Hyperbolic Hierarchical Knowledge Graph Embeddings for Link Prediction in Low Dimensions

Wenjie Zheng, Wenxue Wang, Fulan Qian, Shu Zhao, Yanping Zhang

Abstract

Knowledge graph embeddings (KGE) have been validated as powerful methods for inferring missing links in knowledge graphs (KGs) since they map entities into Euclidean space and treat relations as transformations of entities. Currently, some Euclidean KGE methods model semantic hierarchies prevalent in KGs and promote the performance of link prediction. For hierarchical data, instead of traditional Euclidean space, hyperbolic space as an embedding space has shown the promise of high fidelity and low memory consumption; however, existing hyperbolic KGE methods neglect to model them. To address this issue, we propose a novel KGE model -- hyperbolic hierarchical KGE (HypHKGE). To be specific, we first design the attention-based learnable curvatures for hyperbolic space to preserve rich semantic hierarchies. Moreover, we define the hyperbolic hierarchical transformations based on the theory of hyperbolic geometry, which utilize hierarchies that we preserved to infer the links. Experiments show that HypHKGE can effectively model semantic hierarchies in hyperbolic space and outperforms the state-of-the-art hyperbolic methods, especially in low dimensions.

Introduction

Knowledge Graphs (KGs) like Freebase or WordNet are massive networks of facts, but they are almost always incomplete. Predicting missing links—such as identifying the director of a specific movie—is a core task in modern AI. Most existing models map these graphs into Euclidean space. However, KGs often exhibit "tree-like" or hierarchical structures. For example, a producer like Marvel Studios produces movies, which are then directed by specific individuals, creating a clear vertical hierarchy.

Hierarchical structure in a Knowledge Graph showing the relationship between producers, movies, and directors.

Euclidean space is mathematically "flat," meaning it requires very high dimensions to represent these hierarchies without massive distortion. Hyperbolic space, which has constant negative curvature, is naturally suited for trees because its volume grows exponentially with radius—matching the growth of nodes in a hierarchical tree.

Methodology: The HypHKGE Framework

The proposed HypHKGE model introduces three key innovations to better capture these structures:

  1. Attention-based Learnable Curvatures: Instead of a fixed curvature, the model uses an attention mechanism to calculate a unique curvature based on the head entity and the relation. This adapts the "shape" of the space to the specific local hierarchy of a fact.
  2. Hyperbolic Inter-Level Transformations: To model vertical movement (e.g., from a specific species to a broader genus), the model performs scaling operations in the tangent space—a flat Euclidean space that "touches" the hyperbolic manifold at a point—before projecting back.
  3. Hyperbolic Intra-Level Transformations: To model relationships between peers (e.g., "is_friend"), the model uses Givens rotations. These capture logical patterns like symmetry without changing the entity's hierarchical depth.
The relationship between the hyperbolic manifold and the Euclidean tangent space via exponential and logarithmic maps.

The model utilizes the Poincaré ball model of hyperbolic geometry and relies on exponential and logarithmic maps to move data between the hyperbolic manifold and the Euclidean tangent space where operations are computationally efficient.

Experiments and Key Results

The model was evaluated on standard benchmarks including WN18RR (lexical relations), FB15K-237 (general knowledge), and YAGO3-10 (people and attributes).

Key findings include:

  • Efficiency in Low Dimensions: HypHKGE performs better at 32 dimensions than many Euclidean models do at 500+ dimensions. On YAGO3-10, it improved the Mean Reciprocal Rank (MRR) by 7.6% over the previous best hyperbolic model (AttH).
  • Hierarchy Handling: In specific hierarchical relations like member_meronym, HypHKGE showed improvement rates of up to 24.9% compared to Euclidean counterparts.
  • Ablation Success: Removing either the learnable curvature or the hierarchical transformations led to a significant drop in performance, proving both are necessary to capture the graph's structure.

Conclusion

HypHKGE demonstrates that the degree of hierarchy in a KG is contextual, depending on both relations and specific entities. By utilizing learnable curvatures and specialized transformations, the model achieves state-of-the-art results with significantly lower memory requirements. Future work may explore mixed-curvature spaces to better handle graphs containing both hierarchical and cyclical components.

Summary updated

Abstract

Knowledge graph embeddings (KGE) have been validated as powerful methods for inferring missing links in knowledge graphs (KGs) since they map entities into Euclidean space and treat relations as transformations of entities. Currently, some Euclidean KGE methods model semantic hierarchies prevalent in KGs and promote the performance of link prediction. For hierarchical data, instead of traditional Euclidean space, hyperbolic space as an embedding space has shown the promise of high fidelity and low memory consumption; however, existing hyperbolic KGE methods neglect to model them. To address this issue, we propose a novel KGE model -- hyperbolic hierarchical KGE (HypHKGE). To be specific, we first design the attention-based learnable curvatures for hyperbolic space to preserve rich semantic hierarchies. Moreover, we define the hyperbolic hierarchical transformations based on the theory of hyperbolic geometry, which utilize hierarchies that we preserved to infer the links. Experiments show that HypHKGE can effectively model semantic hierarchies in hyperbolic space and outperforms the state-of-the-art hyperbolic methods, especially in low dimensions.
BibTeX
@misc{zheng2022hyperbolic,
  author = {Wenjie Zheng and Wenxue Wang and Fulan Qian and Shu Zhao and Yanping Zhang},
  title = {{Hyperbolic Hierarchical Knowledge Graph Embeddings for Link Prediction in Low Dimensions}},
  year = {2022},
  eprint = {2204.13704},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2204.13704},
}
Figure