Beyond Transduction: A Survey on Inductive, Few Shot, and Zero Shot Link Prediction in Knowledge Graphs

Nicolas Hubert, Pierre Monnin, Heiko PaulheimarXiv 2023 · arXiv

TL;DR

This paper provides the first comprehensive, systematic review of link prediction in Knowledge Graphs (KGs) beyond the traditional transductive setting. By analyzing 129 papers, the authors expose a chaotic landscape of conflicting terminologies and task definitions, and propose a elegant, "anchor-based" boolean nomenclature to standardize future research.

What Problem Does This Paper Address?

Knowledge graphs are powerful tools for representing structured data, but they are notoriously incomplete. To fix this, researchers use Link Prediction (LP) to infer missing facts. Historically, this was done in a transductive setting, meaning the model assumes all entities and relations in the test set were already seen during training.

Recently, the community has shifted toward more realistic, "knowledge-scarce" scenarios:

  • Inductive LP (ILP): Predicting links for entirely new entities.
  • Few-Shot LP (FSLP): Predicting links for rare, long-tail relations with very few examples.
  • Zero-Shot LP (ZSLP): Predicting links for relations with no training examples at all.

The problem? As these sub-fields emerged, they triggered a "terminological drift." Different researchers started using the same terms to describe completely different experimental setups, or conversely, different terms for the exact same task. This lack of a unified framework makes it nearly impossible to fairly compare new models or understand how these settings truly relate to one another.

Methods

The authors tackle this problem by conducting a rigorous Systematic Literature Review (SLR). Using a highly refined search query across major digital libraries, they whittled down 1,161 papers to a core set of 129 relevant studies.

Fig. 2. Systematic paper selection flowchart

To resolve the terminology crisis, they introduce a novel anchor-based nomenclature. Instead of relying on ambiguous text definitions, they classify tasks using a simple boolean triplet notation: $I_{xyz}$, where $x$, $y$, and $z$ represent the head entity, the relation, and the tail entity, respectively.

  • A value of 1 means the element was seen during training (acting as an "anchor").
  • A value of 0 means the element is unseen.

For example, a standard transductive test triple is classified as $I_{111}$, while a zero-shot relation prediction task between known entities is classified as $I_{101}$.

Datasets

The survey highlights a massive disparity in dataset standardization across settings:

  • FSLP & ZSLP: These fields enjoy relatively stable public benchmarks, primarily relying on NELL-One/Wiki-One (for few-shot) and NELL-ZS/Wiki-ZS (for zero-shot).
  • ILP: This setting is highly fragmented. Most benchmarks are custom-sampled subsets derived from standard KGs like FB15k-237, NELL-995, and WN18RR. Because different authors use wildly different data filtering techniques and train/val/test splits, direct model comparison in ILP has become virtually impossible.

Key Insights and Ideas

  • Explosive Research Growth: Interest in non-transductive link prediction has skyrocketed, with the number of published papers growing exponentially since 2019.

    Fig. 3. Paper count over the years
  • The ILP Definition Mess: The survey reveals that Inductive Link Prediction is the primary culprit for terminological confusion. Some authors define "fully-inductive" as triples containing only unseen entities, while others allow a mix of seen and unseen entities, leading to directly contradictory definitions across papers.

  • The Unexplored Frontier ($I_{000}$): By mapping the literature to their new boolean nomenclature, the authors discovered that the $I_{000}$ setting—where the test triple consists of an unseen head, an unseen relation, and an unseen tail—is completely unstudied as an independent task.

Key Results

The authors successfully mapped all 129 surveyed papers to their new anchor-based nomenclature, revealing exactly where the community's efforts are concentrated:

  • FSLP works consistently map to the $I_{111}$ signature at test time because they rely on a "support set" to learn shallow representations during meta-training.
  • ZSLP works uniformly map to the $I_{101}$ signature (unseen relations, seen entities).
  • ILP works are scattered across a messy combination of configurations. While 34 papers strictly evaluate the $I_{010}$ setting (unseen entities, seen relations), many others evaluate a blend of multiple settings (e.g., 11 papers mix $I_{110} \cup I_{011} \cup I_{010}$), proving that many "inductive" models are actually being evaluated on fundamentally different tasks.

Limitations and Future Work

The authors identify several critical gaps that the community must address:

  1. Standardization is Urgently Needed: Researchers must stop using disparate, custom-filtered datasets for ILP and adopt a unified nomenclature (like the proposed anchor-based framework) to ensure evaluations are actually apples-to-apples.
  2. Arbitrary Hyperparameters: In Few-Shot LP, the choice of $K$ (the number of shots) is often arbitrary, raising concerns about how well these models generalize.
  3. Tackling the Hardest Settings: Future work should push past the comfort zone of evaluating entities and relations in isolation and begin exploring more complex, real-world scenarios—such as the completely unobserved $I_{000}$ setting—where models must handle entirely new entities and relations simultaneously.

Summary updated

Abstract

Knowledge graphs (KGs) comprise entities interconnected by relations of different semantic meanings. KGs are being used in a wide range of applications. However, they inherently suffer from incompleteness, i.e. entities or facts about entities are missing. Consequently, a larger body of works focuses on the completion of missing information in KGs, which is commonly referred to as link prediction (LP). This task has traditionally and extensively been studied in the transductive setting, where all entities and relations in the testing set are observed during training. Recently, several works have tackled the LP task under more challenging settings, where entities and relations in the test set may be unobserved during training, or appear in only a few facts. These works are known as inductive, few-shot, and zero-shot link prediction. In this work, we conduct a systematic review of existing works in this area. A thorough analysis leads us to point out the undesirable existence of diverging terminologies and task definitions for the aforementioned settings, which further limits the possibility of comparison between recent works. We consequently aim at dissecting each setting thoroughly, attempting to reveal its intrinsic characteristics. A unifying nomenclature is ultimately proposed to refer to each of them in a simple and consistent manner.
BibTeX
@misc{hubert2023beyond,
  author = {Nicolas Hubert and Pierre Monnin and Heiko Paulheim},
  title = {{Beyond Transduction: A Survey on Inductive, Few Shot, and Zero Shot Link Prediction in Knowledge Graphs}},
  year = {2023},
  eprint = {2312.04997},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2312.04997},
}
Figure