Trans4E: Link Prediction on Scholarly Knowledge Graphs

Mojtaba Nayyeri, Gokce Muge Cil, Sahar Vahdati, Francesco Osborne, Mahfuzur Rahman, Simone Angioni, Angelo Salatino, Diego Reforgiato Recupero, Nadezhda Vassilyeva, Enrico Motta, Jens LehmannarXiv 2021 · DOI · arXiv

TL;DR

Trans4E is a novel knowledge graph embedding model specifically engineered to handle "many-to-few" relations, such as millions of research papers categorized into a handful of affiliation types. By operating in quaternion space and combining rotations with translations, it achieves state-of-the-art link prediction accuracy even at very low embedding dimensions, making it ideal for massive, incomplete scholarly datasets.

What Problem Does This Paper Address?

Knowledge Graphs (KGs) are powerful tools for representing complex data, but they are almost always incomplete. In the scholarly domain, KGs like the Microsoft Academic Graph (MAG) or AIDA track millions of publications, authors, and topics. However, critical metadata—like whether a paper was produced by a company or a university—is often missing.

The authors identify a specific structural challenge in these KGs: $N$ to $M$ relations where $N \gg M$. For example, in the AIDA graph, over 5 million articles (the "many") are linked to just 8 affiliation types like 'Education' or 'Company' (the "few").

Standard embedding models like TransE or RotatE struggle here. To satisfy their mathematical constraints, they end up mapping all those millions of distinct articles to nearly identical vector positions. This "collapse" makes it impossible for the model to distinguish between different papers, leading to poor performance in predicting missing links. Furthermore, large-scale KGs require low-dimensional embeddings to remain computationally feasible, a setting where existing models typically fail.

Table 1 AIDA - Number of documents associated with the main GRID types.

Methods

The researchers propose Trans4E, which embeds entities and relations in Quaternion space ($\mathbb{H}^d$). Quaternions are a number system that extends complex numbers, allowing for more expressive geometric transformations.

The core innovation is a multi-step scoring function for a triple (head, relation, tail):

  1. Rotation: The head entity vector is rotated in quaternion space by a relation-specific rotation.
  2. Translation: The result is then translated by a relation embedding vector.
  3. Regularization: A mapping regularizer is applied to the tail side to ensure the transformed head meets the tail precisely.

By combining rotation and translation, Trans4E significantly expands the "solution space." Mathematically, the authors show that for a given dimension $d$, Trans4E provides $8^d - 1$ possible distinct vectors for entities involved in these lopsided relations, far exceeding the capacity of previous models.

Datasets

The model was evaluated on:

  • AIDA+MAG: A massive scholarly KG combining the Academia/Industry DynAmics (AIDA) graph and Microsoft Academic Graph. The focus was on completing hasGRIDType (affiliation) and hasTopic (research field) relations.
  • Standard Benchmarks: To prove generalizability, they tested on four classic link prediction datasets: FB15k, FB15k-237, WN18, and WN18RR.

Key Insights and Ideas

  • The "Many-to-Few" Bottleneck: The primary "aha" moment is recognizing that link prediction fails not just because of data size, but because of the ratio of entities in a relation. When millions of heads point to a few tails, the model's geometry must be flexible enough to keep those millions of heads distinct.
  • Quaternion Expressivity: Moving from real or complex numbers to quaternions isn't just a math flex; it provides the necessary degrees of freedom to perform complex rotations and translations simultaneously in a compact vector space.
  • Efficiency at Scale: Scholarly KGs are too big for 1000-dimensional vectors. Trans4E is designed to work at $d=5$ or $d=50$, where it maintains high discriminative power that other models lose.
Figure 1: Distribution of the main topics in academia and industry.

Key Results

Trans4E's performance in low-dimensional settings is striking:

  • AIDA Affiliation Prediction ($d=5$): Trans4E variants achieved over 91% Hits@1 (correctly identifying the affiliation type on the first try). In contrast, the popular TransE model managed only 50%, and RotatE failed completely (0%).
  • Topic Prediction ($d=5$): Trans4E variants reached ~32% Hits@1, while all competing models stayed below 3%.
  • Benchmark Performance: On standard datasets like FB15k and WN18, Trans4E remained highly competitive or superior, particularly in Mean Rank (MR), indicating it is a robust general-purpose model, not just a niche solution for scholarly data.
  • Qualitative Success: t-SNE visualizations of the learned embeddings showed that papers naturally clustered by both topic and publication year, confirming the model captured meaningful semantic relationships.

Limitations and Future Work

While Trans4E dominates in low-dimensional settings, its performance advantage over state-of-the-art models like QuatE or RotatE narrows as the embedding dimension increases (e.g., at $d=500$).

The authors plan to extend this work by:

  • Applying Trans4E to other relation types within AIDA and MAG.
  • Using the completed KGs for downstream tasks like trend detection, expert search, and personalized paper recommendations.
  • Testing the model on other specialized scholarly graphs like the Open Research Knowledge Graph (ORKG).

Summary updated

Abstract

The incompleteness of Knowledge Graphs (KGs) is a crucial issue affecting the quality of AI-based services. In the scholarly domain, KGs describing research publications typically lack important information, hindering our ability to analyse and predict research dynamics. In recent years, link prediction approaches based on Knowledge Graph Embedding models became the first aid for this issue. In this work, we present Trans4E, a novel embedding model that is particularly fit for KGs which include N to M relations with N$\gg$M. This is typical for KGs that categorize a large number of entities (e.g., research articles, patents, persons) according to a relatively small set of categories. Trans4E was applied on two large-scale knowledge graphs, the Academia/Industry DynAmics (AIDA) and Microsoft Academic Graph (MAG), for completing the information about Fields of Study (e.g., 'neural networks', 'machine learning', 'artificial intelligence'), and affiliation types (e.g., 'education', 'company', 'government'), improving the scope and accuracy of the resulting data. We evaluated our approach against alternative solutions on AIDA, MAG, and four other benchmarks (FB15k, FB15k-237, WN18, and WN18RR). Trans4E outperforms the other models when using low embedding dimensions and obtains competitive results in high dimensions.
BibTeX
@misc{nayyeri2021trans,
  author = {Mojtaba Nayyeri and Gokce Muge Cil and Sahar Vahdati and Francesco Osborne and Mahfuzur Rahman and Simone Angioni and Angelo Salatino and Diego Reforgiato Recupero and Nadezhda Vassilyeva and Enrico Motta and Jens Lehmann},
  title = {{Trans4E: Link Prediction on Scholarly Knowledge Graphs}},
  year = {2021},
  eprint = {2107.03297},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2107.03297},
}

Loading related content...

Figure