QuatRE: Relation-Aware Quaternions for Knowledge Graph Embeddings

Dai Quoc Nguyen, Thanh Vu, Tu Dinh Nguyen, Dinh Phung

TL;DR

This paper introduces QuatRE, a novel knowledge graph embedding model that leverages quaternion representations and introduces "relation-aware rotations" to better capture the complex relationships between entities. QuatRE achieves state-of-the-art performance on knowledge graph completion tasks, demonstrating its effectiveness in understanding and predicting relationships.

What Problem Does This Paper Address?

Knowledge graphs (KGs) are powerful tools for representing relationships between entities, often in the form of (head, relation, tail) triples. However, many real-world KGs are incomplete, meaning many valid triples are missing. This necessitates methods to predict these missing links, a task known as knowledge graph completion. Existing embedding models, even those using advanced mathematical spaces like complex numbers or quaternions, often struggle to fully capture the nuanced correlations between head and tail entities for a given relation. For instance, understanding the relationship "has positive test" between "Donald Trump" and "COVID-19" requires considering attributes of both entities, which simpler models might overlook. While some prior models attempted to address this with translation matrices, they significantly increased model complexity and parameters.

Methods

QuatRE builds upon quaternion embeddings, a hyper-complex number system, to represent entities and relations. The core innovation lies in introducing relation-aware rotations. For each relation, QuatRE learns two specific quaternion vectors. These vectors are then used to "rotate" the quaternion embeddings of the head and tail entities. This rotation mechanism, implemented using the Hamilton product, is designed to explicitly enhance the correlations between the head and tail entities based on the specific relation. This approach is presented as a more efficient alternative to the translation matrices used in some previous models, leading to reduced computational overhead. The model's score function is then defined based on these rotated embeddings.

Datasets

The paper evaluates QuatRE on four well-known benchmark datasets for knowledge graph completion:

  • WN18: A subset of WordNet, known for its rich relational structure.
  • FB15k: A subset of Freebase, another large-scale knowledge graph.
  • WN18RR: A subset of WN18 designed to mitigate issues with reversible relations, making the task more challenging.
  • FB15k-237: A subset of FB15k, also curated to address the problem of reversible relations.

Key Insights and Ideas

  • Quaternion Embeddings are Powerful: The paper reinforces the utility of hyper-complex number systems like quaternions for representing complex relational data in KGs.
  • Relation-Aware Rotations are Key: The introduction of relation-specific rotations is the central contribution. This mechanism allows the model to dynamically adjust entity embeddings based on the relation, leading to a richer understanding of their interplay.
  • Efficiency Gains: By replacing translation matrices with quaternion rotations, QuatRE offers a more computationally efficient approach to capturing relational nuances, which is crucial for large-scale KGs.
  • QuatRE as an Extension of QuatE: The paper demonstrates that QuatRE can be seen as a generalization of the existing QuatE model, suggesting a clear lineage and evolutionary step in quaternion-based KG embeddings.

Key Results

QuatRE achieved state-of-the-art performance on several benchmark datasets. Specifically:

  • On WN18, QuatRE achieved the best scores for MR (Mean Rank) and Hits@10, and competitive scores for other metrics.
  • On FB15k, it secured the best scores for MRR (Mean Reciprocal Rank), Hits@3, and Hits@1.
  • On the more challenging datasets WN18RR and FB15k-237, QuatRE outperformed existing baselines across most metrics. For instance, on WN18RR, it showed significant improvements in MR (around 14% relative improvement) and absolute gains in Hits@10 and Hits@3 compared to QuatE.
  • Ablation studies confirmed that both relation-aware rotation vectors (vr,1 and vr,2) contribute positively, and even individual variants outperform the baseline QuatE.

Limitations and Future Work

  • Limitations: The paper does not explicitly detail limitations beyond the inherent challenges of knowledge graph completion. The visualization in Figure 1, while showing improved correlations, is a qualitative assessment.
  • Future Work: The paper positions QuatRE as a strong baseline for future research. The authors suggest its effectiveness in modeling different types of relations (one-to-one, one-to-many, etc.) as a direction for further exploration. The model's ability to handle complex relational structures makes it a promising candidate for more advanced KG tasks.

Summary updated

Abstract

We propose a simple yet effective embedding model to learn quaternion embeddings for entities and relations in knowledge graphs. Our model aims to enhance correlations between head and tail entities given a relation within the Quaternion space with Hamilton product. The model achieves this goal by further associating each relation with two relation-aware rotations, which are used to rotate quaternion embeddings of the head and tail entities, respectively. Experimental results show that our proposed model produces state-of-the-art performances on well-known benchmark datasets for knowledge graph completion. Our code is available at: \url{https://github.com/daiquocnguyen/QuatRE}.
BibTeX
@inproceedings{nguyen2020quatre,
  author = {Dai Quoc Nguyen and Thanh Vu and Tu Dinh Nguyen and Dinh Phung},
  title = {{QuatRE: Relation-Aware Quaternions for Knowledge Graph Embeddings}},
  booktitle = {The Web Conference},
  year = {2020},
  url = {https://openreview.net/forum?id=OyCeSlugve9},
}
Figure