HIP Network: Historical Information Passing Network for Extrapolation Reasoning on Temporal Knowledge Graph

Yongquan He, Peng Zhang, Luchen Liu, Qi Liang, Wenyuan Zhang, Chuang Zhang

Abstract

In recent years, temporal knowledge graph (TKG) reasoning has received significant attention. Most existing methods assume that all timestamps and corresponding graphs are available during training, which makes it difficult to predict future events. To address this issue, recent works learn to infer future events based on historical information. However, these methods do not comprehensively consider the latent patterns behind temporal changes to pass historical information selectively, update representations appropriately, and predict events accurately. In this paper, we propose the Historical Information Passing (HIP) network to predict future events. The HIP network passes information from temporal, structural, and repetitive perspectives, which are used to model the temporal evolution of events, the interactions of events at the same time step, and the known events respectively. In particular, our method considers the updating of relation representations and adopts three scoring functions corresponding to the above dimensions. Experimental results on five benchmark datasets show the superiority of the HIP network, and the significant improvements on Hits@1 prove that our method can more accurately predict what is going to happen.

Introduction: The Challenge of Extrapolation

Traditional Knowledge Graphs (KGs) are static collections of facts represented as triples: (Subject, Relation, Object). However, the real world is dynamic. Temporal Knowledge Graphs (TKGs) add a timestamp to these facts, creating quadruples like (Obama, visit, China, 2014).

The primary challenge in this field is extrapolation reasoning: predicting facts at future timestamps where the graph structure is completely unknown. Most existing models either treat relations as static or fail to capture the complex, overlapping patterns of how events evolve. For example, a diplomatic event might be preceded by policy discussions (temporal evolution), occur alongside trade agreements (structural interaction), or happen periodically (repetitive pattern).

An example of temporal knowledge graph reasoning where future events are predicted based on historical context.

The HIP Network Architecture

The HIP Network architecture consists of three specialized modules designed to disentangle and process these historical patterns.

1. Structural Information Passing (SIP) Module

This module captures how entities interact with their neighbors at a specific point in time. It uses a modified CompGCN (Composition-based Graph Convolutional Network) to update entity and relation embeddings. Crucially, it "disentangles" these embeddings into multiple independent components (channels) to selectively pass information, preventing the "information wash-out" often seen in standard graph convolutions.

2. Temporal Information Passing (TIP) Module

This module focuses on how relations between specific entity pairs change over time. It employs a temporal self-attention layer to weigh the importance of past time steps and a Gated Recurrent Unit (GRU) to model the sequential evolution of these relationships.

3. History Forward (HF) Module

The HF module is the "reasoning engine." It uses three distinct scoring functions to evaluate potential future events:

  • Structural Score: Based on current neighborhood interactions.
  • Temporal Score: Based on the historical evolution of entity pairs.
  • Repetitive Score: Based on a "historical vocabulary" that tracks how often specific facts have occurred in the past.
The architecture of the HIP Network, featuring Structural Information Passing (SIP), Temporal Information Passing (TIP), and History Forward (HF) modules.

Experimental Evaluation

The authors evaluated the HIP Network on five benchmark TKG datasets: ICEWS14, ICEWS18, GDELT, WIKI, and YAGO. These datasets cover a range of temporal granularities, from 15-minute intervals in GDELT to yearly gaps in WIKI/YAGO.

Key Results

The HIP Network achieved state-of-the-art (SOTA) results across all five datasets. The most striking result was the improvement in Hits@1 (the probability that the top-ranked prediction is correct). On the ICEWS18 dataset, HIP achieved a Hits@1 of 43.51, significantly higher than the previous SOTA, CyGNet (42.02). This performance proves that its multi-perspective approach is more effective for the specific challenges of extrapolation.

Conclusion

The HIP Network demonstrates that disentangling structural, temporal, and repetitive patterns is key to accurate future event prediction. By explicitly updating relation representations and accounting for cyclical historical events, the model provides a robust framework for TKG reasoning. Future work may involve reducing the computational complexity of the multi-step reasoning process and incorporating external knowledge like event text descriptions.

Summary updated

Abstract

In recent years, temporal knowledge graph (TKG) reasoning has received significant attention. Most existing methods assume that all timestamps and corresponding graphs are available during training, which makes it difficult to predict future events. To address this issue, recent works learn to infer future events based on historical information. However, these methods do not comprehensively consider the latent patterns behind temporal changes, to pass historical information selectively, update representations appropriately and predict events accurately. In this paper, we propose the Historical Information Passing (HIP) network to predict future events. HIP network passes information from temporal, structural and repetitive perspectives, which are used to model the temporal evolution of events, the interactions of events at the same time step, and the known events respectively. In particular, our method considers the updating of relation representations and adopts three scoring functions corresponding to the above dimensions. Experimental results on five benchmark datasets show the superiority of HIP network, and the significant improvements on Hits@1 prove that our method can more accurately predict what is going to happen.
BibTeX
@inproceedings{he2021hip,
  author = {Yongquan He and Peng Zhang and Luchen Liu and Qi Liang and Wenyuan Zhang and Chuang Zhang},
  title = {{HIP Network: Historical Information Passing Network for Extrapolation Reasoning on Temporal Knowledge Graph}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.24963/ijcai.2021/264},
}
Figure