Mitigating Heterogeneity among Factor Tensors via Lie Group Manifolds for Tensor Decomposition Based Temporal Knowledge Graph Embedding

Jiang Li, Xiangdong Su, Yeyun Gong, Guanglai Gao

Introduction

Temporal Knowledge Graphs (TKGs) represent facts as quadruples $(s, r, o, t)$, adding a temporal dimension to the traditional subject-relation-object triple. To predict missing links in these dynamic structures, researchers frequently employ tensor decomposition. This process breaks down the 4D graph into "factor tensors"—vectors representing entities, relations, and timestamps. However, a fundamental challenge persists: the inherent heterogeneity between these different data types.

The Challenge of Heterogeneity

In TKGs, entities (e.g., "United Nations") are often relatively static, whereas relations ("negotiates with") and timestamps ("2024-Q1") represent dynamic interactions and specific points in time. Mathematically, this discrepancy causes the embeddings for entities, relations, and timestamps to follow vastly different distributions. When a model attempts to fuse these mismatched tensors to calculate a score for a potential fact, the performance is bottlenecked by this mathematical "friction."

Conceptual illustration of heterogeneity (a) vs. homogeneity (b) in factor tensors across entities, relations, and timestamps.

As shown in the figure above, standard models (a) result in scattered, heterogeneous distributions. The proposed method aims to achieve the state shown in (b), where the distributions are aligned and homogeneous, allowing for more effective interaction between the components.

Smoothing via Lie Group Manifolds

To mitigate this heterogeneity, the authors introduce a "plug-and-play" method that maps factor tensors onto a smooth Lie group manifold, specifically the special orthogonal group $SO(n)$ consisting of rotation matrices. Lie groups are mathematically uniform spaces where different types of data can coexist more harmoniously.

Because Lie groups are curved (non-Euclidean), standard optimization techniques cannot be applied directly. The authors utilize a logarithmic map to project these representations into the Lie algebra ($so(n)$), a flat Euclidean space of skew-symmetric matrices where optimization is feasible.

The geometric mapping process from the Lie group manifold SO(n) to the tangent space at identity (Lie algebra so(n)) via the logarithmic map.

This approach is integrated into the model's loss function using N3 regularization, incentivizing the model to learn embeddings that conform to these smooth representations. Remarkably, this method achieves these gains with zero-parameter overhead, as it modifies the training objective rather than adding new weights to the architecture.

Key Insights and Theoretical Foundation

The researchers provide a theoretical proof (Proposition 1) demonstrating that homogeneous factor tensors can approximate ground truth more effectively at a lower rank than heterogeneous ones. By standardizing the semantic roles of entities, relations, and time through the $SO(n)$ manifold, the model requires less complexity to achieve high accuracy.

Experimental Results

The method was evaluated on three major TKG benchmarks: ICEWS14, ICEWS05-15, and GDELT. The results across four state-of-the-art models (TComplEx, TNTComplEx, TeLM, and TeAST) showed consistent improvements:

  • Significant Accuracy Gains: On the ICEWS05-15 dataset, the TeAST model saw its Mean Reciprocal Rank (MRR) increase by 10.4 points and its Hits@1 jump by 12.1 points.
  • Distribution Alignment: t-SNE visualizations and Frobenius norm measurements confirmed that the embeddings for entities and relations became significantly more overlapping and unified.
  • Scalability: The method maintained performance on the massive GDELT dataset (2M quadruples), proving its viability for large-scale applications.

Conclusion

By leveraging the geometric properties of Lie group manifolds, this research provides a robust solution to the heterogeneity problem in temporal knowledge graph embedding. The approach successfully "smooths" the interaction between static entities and dynamic temporal data, leading to state-of-the-art performance without increasing model complexity. Future work may explore other manifold types or apply this homogenization logic to multi-modal graph data.

Summary updated

Abstract

Recent studies have highlighted the effectiveness of tensor decomposition methods in the Temporal Knowledge Graphs Embedding (TKGE) task. However, we found that inherent heterogeneity among factor tensors in tensor decomposition significantly hinders the tensor fusion process and further limits the performance of link prediction. To overcome this limitation, we introduce a novel method that maps factor tensors onto a unified smooth Lie group manifold to make the distribution of factor tensors approximating homogeneous in tensor decomposition. We provide the theoretical proof of our motivation that homogeneous tensors are more effective than heterogeneous tensors in tensor fusion and approximating the target for tensor decomposition based TKGE methods. The proposed method can be directly integrated into existing tensor decomposition based TKGE methods without introducing extra parameters. Extensive experiments demonstrate the effectiveness of our method in mitigating the heterogeneity and in enhancing the tensor decomposition based TKGE models.
BibTeX
@inproceedings{li2025mitigating,
  author = {Jiang Li and Xiangdong Su and Yeyun Gong and Guanglai Gao},
  title = {{Mitigating Heterogeneity among Factor Tensors via Lie Group Manifolds for Tensor Decomposition Based Temporal Knowledge Graph Embedding}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2025},
  url = {https://openreview.net/forum?id=J679JBfC1N},
}
Figure