Bending the Future: Autoregressive Modeling of Temporal Knowledge Graphs in Curvature-Variable Hyperbolic Spaces

Jihoon Sohn, Mingyu Derek Ma, Muhao Chen

TL;DR

HyperVC is a new approach to forecasting future events in Temporal Knowledge Graphs (TKGs) by ditching "flat" Euclidean math for the "curved" geometry of hyperbolic space. By dynamically adjusting the curvature of these mathematical spaces based on how hierarchical the data is at any given moment, the model achieves state-of-the-art performance in predicting future relations, especially in highly structured datasets.

What Problem Does This Paper Address?

Knowledge Graphs (KGs) represent facts as networks of entities and relations (e.g., Joe Biden — IsPresidentOf — U.S.). However, the real world is dynamic; facts change over time. Temporal Knowledge Graphs (TKGs) add a timestamp to these facts, creating a sequence of snapshots.

The challenge lies in extrapolation: predicting what will happen at a future time $t > T$ based on history. Most existing models embed these graphs in Euclidean space. The problem? Euclidean space is "flat," while real-world knowledge is often hierarchical or tree-like. Furthermore, the "tree-likeness" of a graph isn't constant—it changes over time. A model that treats every point in time with the same geometric assumptions misses out on the evolving structural complexity of the data.

Methods

The authors introduce HyperVC (Hyperbolic model with Variable Curvature). Hyperbolic space is often described as a "continuous version of a tree" because its volume grows exponentially, making it perfect for embedding hierarchies with low error.

Key components include:

  • Dual Representations: The model learns Global representations (summarizing an entire snapshot's "vibe" or trend) and Local representations (focusing on specific entity-relation pairs).
  • Variable Curvature ($c_t$): This is the "secret sauce." Instead of a fixed curvature, HyperVC treats curvature as a learnable variable. It models $c_t$ as an additive time series (capturing seasonal trends) or as a function of the Krackhardt hierarchical score (a metric of how tree-like a graph is).
  • Hyperbolic Architecture: The model uses Hyperbolic Graph Attention Networks (GAT) to aggregate neighborhood info and Hyperbolic RNNs to track how these representations evolve over time.
  • Autoregressive Modeling: It predicts the probability of a future event by decomposing it into a chain of conditional probabilities (Subject $\rightarrow$ Relation $\rightarrow$ Object).

Datasets

The model was evaluated on four standard TKG benchmarks:

  1. YAGO: A subset of YAGO3 (Wikipedia-based), known for high hierarchy.
  2. WIKI: Wikipedia history data, also highly hierarchical.
  3. ICEWS18: Integrated Crisis Early Warning System (event data from 2018).
  4. GDELT: Global Database of Events, Language, and Tone (high-frequency news events).

The authors specifically noted that YAGO and WIKI have much higher average Krackhardt hierarchical scores than GDELT or ICEWS18.

Key Insights and Ideas

  • Chronology is a Hierarchy: The authors argue that time itself creates a hierarchy. Events branch out into sub-events, forming an evolutionary path that hyperbolic space captures better than a flat grid.
  • The "Bendiness" Matters: Not all snapshots are created equal. A graph representing a stable political hierarchy needs a different mathematical "bend" (curvature) than a chaotic, highly interconnected set of news events. By making curvature a function of time and hierarchy scores, the model adapts its geometry to the data's shape.
  • Poincaré vs. Lorentz: The paper compares two popular hyperbolic models. They found the Poincaré ball model generally outperformed the Lorentz model for this specific task.

Key Results

HyperVC showed significant improvements in Link Prediction (predicting the missing entity in a fact).

  • Hierarchy Wins: On the highly hierarchical WIKI dataset, HyperVC improved the H@1 (Hits at 1) metric by up to 8.27% compared to RE-Net (a Euclidean autoregressive model).
  • SOTA Performance: On YAGO, HyperVC outperformed the previous State-of-the-Art (HIP Network) in the H@1 metric.
  • Ablation Success: The experiments proved that making curvature a function of time (Equation 2 in the paper) was more effective for datasets with shifting structures (like YAGO) than using a constant curvature.

Limitations and Future Work

The authors noted that while HyperVC excels at capturing structural hierarchy, it doesn't currently include a "historical vocabulary module." Some competing models (like HIP Network) use such modules to specifically track entities that appear frequently in the past. This explains why HyperVC didn't beat the SOTA on the GDELT and ICEWS18 datasets, which are less hierarchical and more driven by repetitive event patterns.

Future Work: The next step is likely integrating historical frequency modules into the hyperbolic framework, combining the "geometric" strengths of HyperVC with the "statistical" strengths of vocabulary-tracking models.

Summary updated

Abstract

Recently there is an increasing scholarly interest in time-varying knowledge graphs, or temporal knowledge graphs (TKG). Previous research suggests diverse approaches to TKG reasoning that uses historical information. However, less attention has been given to the hierarchies within such information at different timestamps. Given that TKG is a sequence of knowledge graphs based on time, the chronology in the sequence derives hierarchies between the graphs. Furthermore, each knowledge graph has its hierarchical level which may differ from one another. To address these hierarchical characteristics in TKG, we propose HyperVC, which utilizes hyperbolic space that better encodes the hierarchies than Euclidean space. The chronological hierarchies between knowledge graphs at different timestamps are represented by embedding the knowledge graphs as vectors in a common hyperbolic space. Additionally, diverse hierarchical levels of knowledge graphs are represented by adjusting the curvatures of hyperbolic embeddings of their entities and relations. Experiments on four benchmark datasets show substantial improvements, especially on the datasets with higher hierarchical levels.
BibTeX
@inproceedings{sohn2022bending,
  author = {Jihoon Sohn and Mingyu Derek Ma and Muhao Chen},
  title = {{Bending the Future: Autoregressive Modeling of Temporal Knowledge Graphs in Curvature-Variable Hyperbolic Spaces}},
  booktitle = {AKBC},
  year = {2022},
  url = {https://openreview.net/forum?id=jBYThB7L1j},
}

Loading related content...

Figure