Introduction
Knowledge Graphs (KGs) traditionally represent facts as triples $(s, r, o)$, such as (Obama, visit, China). Temporal Knowledge Graphs (TKGs) extend this by adding a time dimension, $t$. While much research has focused on interpolation (filling in missing historical data), this paper addresses the more challenging task of extrapolation: forecasting future events where the graph structure is entirely unknown.
Real-world events follow sophisticated patterns that existing models often struggle to capture simultaneously:
- Structural Patterns: Simultaneous events influence one another (e.g., a country considering a policy before making a statement).
- Temporal Patterns: The relationship between specific entities evolves over time.
- Repetitive Patterns: Many events, such as annual summits or diplomatic frictions, are periodic or recur.
The HIP Network (Historical Information Passing) is proposed to disentangle and model these three distinct historical patterns to significantly improve forecasting accuracy.
Methodology
The HIP Network architecture is composed of three specialized modules designed to handle different aspects of historical information passing.
Structural Information Passing (SIP) Module
The SIP module utilizes a modified Graph Convolutional Network (CompGCN) to capture neighborhood interactions at specific timestamps. It updates entity and relation representations in a disentangled manner, allowing the model to selectively pass information about how entities and relations interact at a specific point in time.
Temporal Information Passing (TIP) Module
This module focuses on the evolution of relations between specific entity pairs. It employs a Temporal Self-Attention layer to integrate entity representations across time and a Gated Recurrent Unit (GRU) to model the sequence of historical relations that have occurred between a subject and an object.
History Forward (HF) Module
The HF module serves as the reasoning engine. It uses three distinct scoring functions—Structural, Temporal, and Repetitive—to evaluate the likelihood of future events. For multi-step forecasting, it iteratively predicts events for intermediate timestamps, adding them to a "predicted graph" to maintain context for future steps.
Experimental Setup
The HIP Network was evaluated on five benchmark TKG datasets:
- ICEWS14 & ICEWS18: Event-based data from the Integrated Crisis Early Warning System.
- GDELT: Large-scale news event data from the Global Database of Events, Language, and Tone.
- WIKI & YAGO: Long-term historical facts derived from Wikipedia and YAGO3.
These datasets vary in scale and temporal granularity, ranging from 15-minute intervals (GDELT) to yearly gaps (WIKI/YAGO).
Results and Discussion
The HIP Network achieved state-of-the-art (SOTA) performance across all evaluated datasets.
Key Performance Metrics
A significant improvement was observed in Hits@1 (the probability that the top-ranked prediction is correct). For instance, on the ICEWS18 dataset, HIP achieved a Hits@1 of 43.51, outperforming the previous best baseline, CyGNet (42.02). High Mean Reciprocal Rank (MRR) scores further indicate the model's precision in pinpointing exact future events compared to models relying solely on historical distributions.
Insights
- Relation Dynamics: Unlike models that only update entity embeddings, HIP considers the evolution of relation embeddings, acknowledging that the context of relations like criticize or consult changes over time.
- Disentanglement: Separating structural, temporal, and repetitive components allows the model to adapt to different data availability. For new entities, the structural module provides the primary signal, while the repetitive module handles recurring events.
- Ablation Success: Removing any of the three modules resulted in a performance decline, confirming that all three perspectives are essential for robust extrapolation.
Conclusion and Future Work
While the HIP Network demonstrates superior performance, it is sensitive to the historical window length. Performance typically plateaus after a certain point, and very long sequences increase the computational load of the RNN components. Additionally, the multi-step "hallucination" process is more computationally intensive than one-step models. Future work may involve refining the selection of historical information through advanced gating mechanisms to improve both efficiency and accuracy.
Summary updated