Abstract
Temporal Knowledge Graph Completion (tKGC) has seen a surge in complex models utilizing time-dependent entity embeddings (TEEs). However, the relative contributions of model architecture versus training heuristics have remained unclear. This paper introduces a Composable Unified Framework to systematically decouple these factors. Through an exhaustive evaluation involving over 19,000 GPU hours, the authors reveal a "reality check": simple timestamp embeddings (TEs) often outperform state-of-the-art TEEs, which frequently overfit on sparse temporal data. Furthermore, properly tuned baselines achieve performance levels significantly higher than previously reported, suggesting that much of the recent progress in tKGC is driven by training refinements rather than architectural complexity.
Introduction
Knowledge Graphs (KGs) represent facts as triples $(s, r, o)$. Temporal Knowledge Graphs (tKGs) extend this by adding a temporal dimension, resulting in quadruples $(s, r, o, t)$, such as (Bob, livesIn, New York, 2020). The goal of Temporal Knowledge Graph Completion is to predict missing links within these time-sensitive structures.
As the field has evolved, new models have simultaneously introduced novel score functions, loss functions, and training heuristics. This lack of isolation makes it difficult to determine the true source of performance gains. This research addresses this "muddy" landscape by evaluating models under a unified framework to identify what actually drives state-of-the-art results.
A Composable Unified Framework
To ensure a fair comparison, the researchers developed a framework that treats tKGC models as a combination of six interchangeable modules:
- Temporal Embedding Layer: Representations of time (discrete vs. continuous).
- Static Embedding Layer: Traditional entity and relation representations.
- Score Function: Calculation of fact plausibility (e.g., TransE, SimplE).
- Loss Function: Optimization objectives (e.g., Cross-Entropy).
- Regularization: Techniques to mitigate overfitting.
- Reciprocal Relations: Explicit modeling of inverse relationships.
Using Bayesian optimization, the team conducted over 4,000 experiments to find the optimal hyperparameters for six major temporal embedding approaches across these modules.
Key Insights: The Complexity Trap
Sparsity and Overfitting
A prevailing assumption in tKGC is that Time-dependent Entity Embeddings (TEEs)—where each entity has a unique function describing its evolution—are superior due to their expressiveness. However, the study found that TEEs often struggle because most entities appear infrequently. Attempting to learn complex temporal functions for sparse entities leads to significant overfitting.
To test this, the authors created UTEE (Unique Temporal Entity Embedding), which applies a single temporal function to all entities. UTEE matched or exceeded the performance of entity-specific TEEs, confirming that the additional parameters in TEEs often capture noise rather than meaningful temporal patterns.
The Power of Baselines
When integrated into the unified framework and properly tuned, older models showed dramatic improvements. For instance, T-TransE (2018) achieved a Mean Reciprocal Rank (MRR) of 55.3% on the ICEWS14 dataset—more than double the 25.5% originally reported. This suggests that the perceived superiority of newer models is often a result of better training "tricks" like reciprocal relations and position-aware entity embeddings.
Experimental Results
The models were evaluated on ICEWS14, ICEWS11-14, and GDELT-m10.
On the ICEWS datasets, the simple T embedding (timestamp-only) consistently outperformed more complex models like DE and ATiSE. However, on the GDELT-m10 dataset—which is significantly denser—TEEs did show an advantage. This indicates that the optimal model architecture is highly dependent on the density of the underlying data.
Conclusion
This re-evaluation demonstrates that time-dependent entity embeddings are not always necessary and can even be detrimental in sparse data scenarios. The study highlights the importance of fair benchmarking and suggests that future research should focus on adaptive temporal embeddings that can handle varying levels of data density while maintaining the robustness of simpler baseline architectures.
Summary updated