Bending the Future: Autoregressive Modeling of Temporal Knowledge Graphs in Curvature-Variable Hyperbolic Spaces

Jihoon Sohn, Mingyu Derek Ma, Muhao Chen

Abstract

Recently there is an increasing scholarly interest in time-varying knowledge graphs, or temporal knowledge graphs (TKG). Previous research suggests diverse approaches to TKG reasoning that uses historical information. However, less attention has been given to the hierarchies within such information at different timestamps. Given that TKG is a sequence of knowledge graphs based on time, the chronology in the sequence derives hierarchies between the graphs. Furthermore, each knowledge graph has its hierarchical level which may differ from one another. To address these hierarchical characteristics in TKG, we propose HyperVC, which utilizes hyperbolic space that better encodes the hierarchies than Euclidean space. The chronological hierarchies between knowledge graphs at different timestamps are represented by embedding the knowledge graphs as vectors in a common hyperbolic space. Additionally, diverse hierarchical levels of knowledge graphs are represented by adjusting the curvatures of hyperbolic embeddings of their entities and relations. Experiments on four benchmark datasets show substantial improvements, especially on the datasets with higher hierarchical levels.

Introduction

Knowledge Graphs (KGs) represent facts as networks of entities and relations (e.g., Joe Biden — IsPresidentOf — U.S.). However, real-world knowledge is dynamic, leading to the development of Temporal Knowledge Graphs (TKGs) which add a timestamp to these facts, creating a sequence of snapshots. A primary challenge in TKG research is extrapolation: predicting future events based on historical data.

Most existing models embed these graphs in Euclidean space. However, Euclidean space is "flat," while real-world knowledge often exhibits hierarchical or tree-like structures. Furthermore, the "tree-likeness" of a graph is not static; it evolves over time. A model that assumes a fixed geometry across all timestamps fails to capture the changing structural complexity of the data. To address this, we propose HyperVC (Hyperbolic model with Variable Curvature), which leverages the properties of hyperbolic space to better represent evolving hierarchies.

Methodology

Hyperbolic space is often viewed as a continuous version of a tree because its volume grows exponentially with its radius, making it ideal for embedding hierarchical data with minimal distortion. HyperVC introduces several key innovations to exploit this geometry.

Dual Representations and Variable Curvature

The model learns both Global representations, which summarize the overall trend of a snapshot, and Local representations, which focus on specific entity-relation pairs. The core innovation of HyperVC is Variable Curvature ($c_t$). Instead of using a fixed curvature, HyperVC treats curvature as a learnable variable that adapts to the data. It models $c_t$ either as an additive time series to capture seasonal trends or as a function of the Krackhardt hierarchical score, a metric quantifying how tree-like a graph is at a given moment.

Hyperbolic Architecture

The architecture employs Hyperbolic Graph Attention Networks (GAT) to aggregate neighborhood information and Hyperbolic Recurrent Neural Networks (hRNNs) to track the evolution of these representations over time.

The HyperVC architecture, illustrating the flow from graph snapshots to local and global hyperbolic representations in variable curvature spaces.

The model predicts the probability of a future event by decomposing it into a chain of conditional probabilities (Subject $\rightarrow$ Relation $\rightarrow$ Object), ensuring an autoregressive approach to forecasting.

Geometric Models

The paper explores different models of hyperbolic space, specifically the Poincaré ball and the Lorentz (hyperboloid) model. While both offer the necessary exponential volume growth, empirical results suggest that the Poincaré ball model is particularly effective for this task.

Visualization of the Lorentz and Poincaré models of hyperbolic space, used to embed hierarchical knowledge graph structures.

Experimental Evaluation

HyperVC was evaluated on four standard TKG benchmarks:

  1. YAGO: A subset of Wikipedia-based data with high hierarchical structure.
  2. WIKI: Wikipedia history data, also highly hierarchical.
  3. ICEWS18: Event data from the Integrated Crisis Early Warning System (2018).
  4. GDELT: High-frequency news events from the Global Database of Events, Language, and Tone.

Analysis of these datasets revealed that YAGO and WIKI possess significantly higher average Krackhardt hierarchical scores compared to GDELT or ICEWS18, providing a diverse testing ground for the variable curvature approach.

Results and Discussion

HyperVC demonstrated significant improvements in Link Prediction tasks, particularly on datasets with strong hierarchical characteristics.

  • Impact of Hierarchy: On the highly hierarchical WIKI dataset, HyperVC improved the Hits@1 (H@1) metric by up to 8.27% compared to RE-Net, a state-of-the-art Euclidean autoregressive model.
  • Performance on YAGO: HyperVC outperformed the previous state-of-the-art (HIP Network) on the YAGO dataset in terms of H@1.
  • Curvature Adaptation: Ablation studies confirmed that making curvature a function of time was more effective for datasets with shifting structures (like YAGO) than using a constant curvature.

The results suggest that chronology itself acts as a hierarchy, where events branch out into sub-events over time. Hyperbolic space captures this evolutionary path more naturally than a flat Euclidean grid.

Conclusion

HyperVC successfully demonstrates that adapting the geometry of the embedding space to the hierarchical nature of the data leads to superior TKG forecasting. While the model excels at capturing structural hierarchy, it currently lacks a "historical vocabulary module" to track frequently recurring entities—a feature present in some competing models like HIP Network. This explains the relatively lower performance on GDELT and ICEWS18, which are driven more by repetitive patterns than deep hierarchies. Future work will focus on integrating historical frequency modules into the hyperbolic framework to combine geometric precision with statistical tracking.

Summary updated

Abstract

Recently there is an increasing scholarly interest in time-varying knowledge graphs, or temporal knowledge graphs (TKG). Previous research suggests diverse approaches to TKG reasoning that uses historical information. However, less attention has been given to the hierarchies within such information at different timestamps. Given that TKG is a sequence of knowledge graphs based on time, the chronology in the sequence derives hierarchies between the graphs. Furthermore, each knowledge graph has its hierarchical level which may differ from one another. To address these hierarchical characteristics in TKG, we propose HyperVC, which utilizes hyperbolic space that better encodes the hierarchies than Euclidean space. The chronological hierarchies between knowledge graphs at different timestamps are represented by embedding the knowledge graphs as vectors in a common hyperbolic space. Additionally, diverse hierarchical levels of knowledge graphs are represented by adjusting the curvatures of hyperbolic embeddings of their entities and relations. Experiments on four benchmark datasets show substantial improvements, especially on the datasets with higher hierarchical levels.
BibTeX
@inproceedings{sohn2022bending,
  author = {Jihoon Sohn and Mingyu Derek Ma and Muhao Chen},
  title = {{Bending the Future: Autoregressive Modeling of Temporal Knowledge Graphs in Curvature-Variable Hyperbolic Spaces}},
  booktitle = {AKBC},
  year = {2022},
  url = {https://openreview.net/forum?id=jBYThB7L1j},
}
Figure