Learning Low-dimensional Multi-domain Knowledge Graph Embedding via Dual Archimedean Spirals

Jiang Li, Xiangdong Su, Fujun Zhang, Guanglai GaoACL 2024 · OpenReview

This research, presented at ACL 2024, introduces DuASE (Dual Archimedean Spiral Embedding), a novel framework for Knowledge Graph Embeddings (KGE) designed to handle multi-domain data in low-dimensional spaces. By leveraging the geometric properties of spirals, the model addresses the critical issue of "domain overlapping" in vector space.

The Challenge: Domain Overlap and Dimensionality

Knowledge Graphs (KGs) aggregate facts from diverse domains such as medicine, sports, and cinema. Traditional KGE models often fail to distinguish between these domains, leading to "embedding overlapping" where entities from unrelated fields occupy the same vector space. This lack of clear boundaries degrades link prediction performance because the model loses the contextual specificity of the domain. Furthermore, many high-performing models require high-dimensional vectors (e.g., 500+ dimensions), which significantly increases computational costs.

Methodology: The Dual Archimedean Spiral

The core innovation of DuASE is the use of the Archimedean Spiral (defined by $\phi = a + b\theta$) to represent entities and relations. The authors discovered that relation types serve as excellent proxies for domains. By mapping entities involved in specific relations onto distinct spiral paths, the model naturally separates different domains.

Geometric representation of the Archimedean spiral (a) and the dual spiral mechanism (b) used in DuASE.

As shown in the figure above, the model defines two spirals ($S_h$ and $S_t$) for every triplet (head, relation, tail). The parameter $b$, which controls the distance between loops, allows the model to distinguish between different domains. A specialized regularization function forces the head and tail spirals of a specific triplet to cluster within their respective domain spaces, ensuring that "medical" entities stay near other "medical" entities and far from "sports" entities.

Experimental Evaluation

The researchers evaluated DuASE on two fronts:

  1. $n$-MDKG (New): A Multi-Domain Knowledge Graph dataset with 3, 6, and 9 domain variants created to test how models handle increasing domain complexity.
  2. Standard Benchmarks: Gold-standard datasets including WN18RR, FB15K-237, and YAGO3-10.

Results and Visual Evidence

DuASE consistently outperformed both high-dimensional and low-dimensional baselines (such as TransE, RotatE, and hyperbolic models like RotH) across all datasets. On the 9-MDKG dataset, it achieved a Mean Reciprocal Rank (MRR) of 0.324, significantly higher than RotatE’s 0.212.

t-SNE visualizations comparing domain separation in (a) ComplEx, (b) RotatE, (c) RotH, and (d) DuASE.

Visual analysis via t-SNE (above) confirms that while other models produce a "giant blob" of mixed-up domains, DuASE creates clearly separated clusters for different subject areas. Despite this structural complexity, DuASE remains lightweight; it achieves state-of-the-art results with just 32 dimensions and trains roughly 10 times faster than complex hyperbolic models like RotH.

Conclusion and Future Work

DuASE demonstrates that geometric priors—specifically spirals—can effectively resolve domain overlapping in KGEs without requiring high dimensionality. While the current model is transductive and primarily relies on relations to define domains, it provides a powerful, efficient solution for large-scale, multi-domain knowledge representation. Future work may focus on handling multifaceted entities that belong to multiple domains simultaneously.

Summary updated

Abstract

Knowledge graph embedding (KGE) is extensively employed for link prediction by representing entities and relations as low-dimensional vectors. In real-world scenarios, knowledge graphs (KGs) usually encompass diverse domains, which poses challenges to KG representations. However, existing KGE methods rarely make domain constraints on the embedding distribution of multi-domain KGs, leading to the embedding overlapping of different domains and performance degradation of link prediction. To address this challenge, we propose Dual Archimedean Spiral Knowledge Graph Embedding (DuASE), a low-dimensional KGE model for multi-domain KGs. DuASE is inspired by our discovery that relation types can distinguish entities from different domains. Specifically, DuASE encodes entities with the same relation on the same Archimedean spiral, allowing it to differentiate the entities from different domains. To avoid embedding overlapping across domains, DuASE further makes the head and the tail spirals in the same triplet cluster to their respective domain space by a regularization function. Thus, DuASE can better capture the domain information and the dependencies between entities when modeling the multi-domain KGs, leading to improved KG representations. We validate the effectiveness of DuASE on the novel multi-domain dataset (n-MDKG) introduced in this study and three other benchmark datasets.
BibTeX
@inproceedings{li2024learning,
  author = {Jiang Li and Xiangdong Su and Fujun Zhang and Guanglai Gao},
  title = {{Learning Low-dimensional Multi-domain Knowledge Graph Embedding via Dual Archimedean Spirals}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2024},
  url = {https://openreview.net/forum?id=9IJXkwQqxH},
}
Figure