Introduction
Temporal Knowledge Graphs (TKGs) represent facts as quadruples $(s, r, o, t)$, adding a temporal dimension to the traditional subject-relation-object triple. To predict missing links in these dynamic structures, researchers frequently employ tensor decomposition. This process breaks down the 4D graph into "factor tensors"—vectors representing entities, relations, and timestamps. However, a fundamental challenge persists: the inherent heterogeneity between these different data types.
The Challenge of Heterogeneity
In TKGs, entities (e.g., "United Nations") are often relatively static, whereas relations ("negotiates with") and timestamps ("2024-Q1") represent dynamic interactions and specific points in time. Mathematically, this discrepancy causes the embeddings for entities, relations, and timestamps to follow vastly different distributions. When a model attempts to fuse these mismatched tensors to calculate a score for a potential fact, the performance is bottlenecked by this mathematical "friction."
As shown in the figure above, standard models (a) result in scattered, heterogeneous distributions. The proposed method aims to achieve the state shown in (b), where the distributions are aligned and homogeneous, allowing for more effective interaction between the components.
Smoothing via Lie Group Manifolds
To mitigate this heterogeneity, the authors introduce a "plug-and-play" method that maps factor tensors onto a smooth Lie group manifold, specifically the special orthogonal group $SO(n)$ consisting of rotation matrices. Lie groups are mathematically uniform spaces where different types of data can coexist more harmoniously.
Because Lie groups are curved (non-Euclidean), standard optimization techniques cannot be applied directly. The authors utilize a logarithmic map to project these representations into the Lie algebra ($so(n)$), a flat Euclidean space of skew-symmetric matrices where optimization is feasible.
This approach is integrated into the model's loss function using N3 regularization, incentivizing the model to learn embeddings that conform to these smooth representations. Remarkably, this method achieves these gains with zero-parameter overhead, as it modifies the training objective rather than adding new weights to the architecture.
Key Insights and Theoretical Foundation
The researchers provide a theoretical proof (Proposition 1) demonstrating that homogeneous factor tensors can approximate ground truth more effectively at a lower rank than heterogeneous ones. By standardizing the semantic roles of entities, relations, and time through the $SO(n)$ manifold, the model requires less complexity to achieve high accuracy.
Experimental Results
The method was evaluated on three major TKG benchmarks: ICEWS14, ICEWS05-15, and GDELT. The results across four state-of-the-art models (TComplEx, TNTComplEx, TeLM, and TeAST) showed consistent improvements:
- Significant Accuracy Gains: On the ICEWS05-15 dataset, the TeAST model saw its Mean Reciprocal Rank (MRR) increase by 10.4 points and its Hits@1 jump by 12.1 points.
- Distribution Alignment: t-SNE visualizations and Frobenius norm measurements confirmed that the embeddings for entities and relations became significantly more overlapping and unified.
- Scalability: The method maintained performance on the massive GDELT dataset (2M quadruples), proving its viability for large-scale applications.
Conclusion
By leveraging the geometric properties of Lie group manifolds, this research provides a robust solution to the heterogeneity problem in temporal knowledge graph embedding. The approach successfully "smooths" the interaction between static entities and dynamic temporal data, leading to state-of-the-art performance without increasing model complexity. Future work may explore other manifold types or apply this homogenization logic to multi-modal graph data.
Summary updated