Timespan-Aware Dynamic Knowledge Graph Embedding by Incorporating Temporal Evolution

Xiaoli Tang, Rui Yuan, Qianyu Li, Tengyun Wang, Haizhi Yang, Yundong Cai, Hengjie SongIEEE Access 2020 · DOI

Introduction

Knowledge Graphs (KGs) like YAGO or Wikidata store vast amounts of information as "triples" (e.g., Bill Clinton, wasPresidentOf, USA). Most embedding methods treat these as static truths. However, the world is dynamic: Bill Clinton was president from 1993 to 2001, but not before or after.

Temporal evolution of facts in a dynamic knowledge graph, illustrating how relations like 'wasPresidentOf' change over time.

While some "dynamic" models exist, they suffer from two main flaws:

  1. Independence Bias: They often treat different time periods as independent snapshots, failing to capture the continuous "evolutionary" thread of how entities change.
  2. Time Unbalance: Historical data is messy. One "time bin" might cover 180 years of stable history, while another covers a single year of rapid change. Standard sequence models (like basic RNNs) assume equal steps between observations, which leads to inaccuracies when the actual timespan between data points varies wildly.

The TDG2E Framework

The authors propose TDG2E (Timespan-aware Dynamic knowledge Graph Embedding Evolution). The architecture consists of three main pillars:

1. Temporal Projection

Building on the TransE model, TDG2E projects entities and relations onto time-specific hyperplanes. This allows the same entity to have different mathematical representations at different points in history, reflecting its changing context.

2. GRU-based Evolution

The model uses Gated Recurrent Units (GRUs) to link these hyperplanes across time. To ensure the model doesn't "forget" the structural logic of the graph, they incorporate an auxiliary loss function. This forces the GRU’s hidden state to actively supervise the learning of the next time step's hyperplane, maintaining structural continuity.

3. The Timespan Gate ($T_\tau$)

This is the paper's core innovation. The authors modified the standard GRU cell by adding a gate that specifically ingests the duration ($\Delta t$) between time bins. This gate controls the flow of historical information—essentially "weighting" the influence of the past based on how much time has actually elapsed. If a long time has passed between two snapshots, the "Timespan Gate" allows the model to adjust how much of the previous structural information is still relevant.

Experimental Evaluation

The model was evaluated on two large-scale temporal datasets: YAGO11k (~20.5k triples) and Wikidata12k (~40k triples). To handle the "unbalance" issue, the authors ensured each time bin contained at least 300 triples, resulting in bins with varying chronological spans (from 1 year to over 100 years).

Key Results

TDG2E demonstrated significant improvements over both static (TransE, HolE) and dynamic (t-TransE, HyTE) baselines:

  • Accuracy: On the Wikidata12k dataset, TDG2E improved the Hit@10 (the probability that the correct answer is in the top 10 results) for head and tail entity prediction by 30.05% and 16.55% respectively over the best-performing baseline.
  • Qualitative Success: In case studies, static models often confused birth and death dates because they only saw the co-occurrence of names and places. TDG2E correctly ordered these events by learning the temporal flow.
  • Robustness: The "Timespan Gate" variant (TDG2E) consistently outperformed a version using a standard GRU (TDG2E-T), proving that explicitly modeling the time interval is crucial for performance.

Conclusion and Future Work

TDG2E shifts the paradigm from treating time as a label to treating it as a filter. By incorporating the timespan between events as a gating mechanism, it captures the pulse of historical evolution more accurately than previous methods.

Future work may address the complexity of hyperplanes by incorporating relation-specific information into temporal projections, as well as exploring performance with finer-grained data like days or months.

Summary updated

BibTeX
@article{tang2020timespan,
  author = {Xiaoli Tang and Rui Yuan and Qianyu Li and Tengyun Wang and Haizhi Yang and Yundong Cai and Hengjie Song},
  title = {{Timespan-Aware Dynamic Knowledge Graph Embedding by Incorporating Temporal Evolution}},
  journal = {IEEE Access},
  year = {2020},
  doi = {10.1109/access.2020.2964028},
}
Figure