Few-Shot Inductive Learning on Temporal Knowledge Graphs using Concept-Aware Information

Zifeng Ding, Jingpei Wu, Bailan He, Yunpu Ma, Zhen Han, Volker Tresp

Introduction

Knowledge Graphs (KGs) serve as the backbone for many modern AI applications, representing facts as structured triples such as (Subject, Relation, Object). However, real-world knowledge is rarely static. Temporal Knowledge Graphs (TKGs) address this by adding a time dimension, transforming triples into quadruples: (Subject, Relation, Object, Timestamp). While TKGs more accurately reflect the fluid nature of reality—such as political appointments or evolving diplomatic relations—they face a significant challenge: the "Out-of-Graph" (OOG) entity problem.

Most existing TKG models are transductive, meaning they can only reason about entities seen during the training phase. In practice, new entities emerge constantly, often with very little associated data. This creates a "long-tail" distribution where traditional models struggle to generalize. To bridge this gap, this research introduces FILT (Few-shot Inductive Learning on TKGs), a framework designed to predict facts about brand-new entities using only a handful of observed examples.

The FILT Framework

FILT addresses the inductive learning challenge through three primary components: concept modeling, a time-sensitive graph encoder, and a meta-learning training strategy.

1. Concept-Aware Modeling

One of the key innovations of FILT is the use of "meta-information" or semantic anchors. In datasets like the Integrated Crisis Early Warning System (ICEWS), entities are often categorized into sectors (e.g., Government, Media, NGO). Even if a specific entity like a newly elected official is unknown to the model, their "sector" provides a semantic prior. FILT learns representations for these concepts by aggregating the embeddings of known entities within the same sector. When an unseen entity appears, its associated sector acts as a shortcut, allowing the model to transfer knowledge from similar entities it has encountered before.

2. Time Difference-Based Graph Encoder

In a temporal context, not all historical interactions are equally relevant. FILT utilizes a specialized encoder to represent a new entity based on its "support set" (the small number of observed links). Unlike static models, FILT weights these neighbors based on their temporal proximity to the query. A fact that occurred yesterday is given more weight than one from several years ago. Interestingly, the researchers found that a simple exponential decay function for temporal weighting outperformed more complex attention mechanisms like Time2Vec. This suggests that for few-shot scenarios with sparse neighborhoods (1–5 edges), simpler models are less prone to overfitting.

3. Meta-Learning Strategy

To ensure the model can generalize to truly unseen entities, FILT is trained using a meta-learning (or "learning to learn") approach. During training, the model is presented with simulated OOG tasks. By repeatedly forcing the model to predict missing links based on only 1 or 3 support edges, the framework learns to extract the most salient information from minimal data rather than simply memorizing entity IDs.

Experimental Evaluation

To test the framework, the authors developed three new benchmark datasets derived from ICEWS, specifically partitioned to ensure that test entities were entirely absent during training:

  • ICEWS14-OOG: Data from the year 2014.
  • ICEWS18-OOG: Data from the year 2018.
  • ICEWS0515-OOG: A long-term dataset covering 2005 to 2015.

Key Results

FILT was benchmarked against a variety of models, including traditional KG completion models (ComplEx), TKG models (TeLM, TeRo), and static few-shot models (GEN). The results demonstrated FILT's clear superiority in the inductive setting:

  • Performance Gains: On the ICEWS18-OOG (3-shot) task, FILT achieved a Mean Reciprocal Rank (MRR) of 0.266, significantly outperforming the best static few-shot baseline, GEN (0.216).
  • Ablation Insights: Removing the concept modeling component led to a significant performance drop, confirming that category-level metadata is crucial for inductive reasoning.
  • Scalability: The model showed consistent improvement as the number of available examples (the "shot" size) increased from 1 to 5, indicating effective information aggregation.

Conclusion and Future Directions

FILT demonstrates that by combining temporal decay logic with high-level semantic concepts, models can effectively reason about new entities in dynamic environments. While the current framework focuses on link prediction within a known timeline, it sets the stage for future work in temporal forecasting—predicting events before they happen. Additionally, while the model currently relies on existing metadata (sectors), future iterations could explore automated ways to derive these concepts, further reducing the dependency on manual data labeling.

Summary updated

BibTeX
@inproceedings{ding2022few,
  author = {Zifeng Ding and Jingpei Wu and Bailan He and Yunpu Ma and Zhen Han and Volker Tresp},
  title = {{Few-Shot Inductive Learning on Temporal Knowledge Graphs using Concept-Aware Information}},
  booktitle = {AKBC},
  year = {2022},
  url = {https://openreview.net/forum?id=klj9oYGJKj},
}
Figure