Mitigating Heterogeneity among Factor Tensors via Lie Group Manifolds for Tensor Decomposition Based Temporal Knowledge Graph Embedding

Jiang Li, Xiangdong Su, Guanglai Gao

This research, presented at ACL 2025, addresses a fundamental challenge in modeling time-evolving facts: the mathematical mismatch between entities, relations, and timestamps. By leveraging tools from differential geometry, the authors demonstrate that "smoothing out" the differences between these components leads to more powerful and efficient Temporal Knowledge Graph Embedding (TKGE) models.

The Challenge: Heterogeneity in Temporal Data

Knowledge Graphs (KGs) represent facts as triples: (Subject, Relation, Object). Temporal Knowledge Graphs (TKGs) extend this by adding a fourth dimension: Time. To predict missing facts, researchers typically use tensor decomposition, which breaks down a massive 4D data block into smaller "factor tensors" representing entities, relations, and timestamps.

Conceptual illustration of mitigating heterogeneity among factor tensors.

The primary obstacle is heterogeneity. Entities are static nodes, relations are interactions, and timestamps are dynamic markers. Because they play distinct semantic roles, their mathematical distributions are wildly different. This lack of "fusion" forces models to use much higher mathematical complexity (ranks) to achieve accuracy, leading to inefficiency. The authors provide a theoretical proof showing that if factor tensors share a common set of basis vectors (i.e., they are homogeneous), a model can approximate the ground truth much more effectively using a lower rank.

Proposed Solution: Lie Group Manifolds

The authors introduce a "plug-and-play" method to mitigate this heterogeneity using Lie Group Manifolds, specifically the Special Orthogonal group $SO(n)$.

  1. Mapping to $SO(n)$: They map factor tensors onto $SO(n)$, which consists of rotation matrices. As a Lie group, $SO(n)$ is a "smooth" manifold that looks mathematically identical at every point. Forcing diverse data onto this manifold encourages the distributions to become more similar (homogeneous).
  2. The Logarithmic Map: Since Lie groups are curved, non-Euclidean spaces, standard optimization is difficult. The authors utilize a "Log Map" to project data into the Lie algebra ($so(n)$), a flat Euclidean space where differences and gradients can be easily calculated.
  3. N3 Regularization: Instead of redesigning models from scratch, this mapping is integrated into existing state-of-the-art architectures (like TComplEx or TeAST) via N3 regularization. This minimizes the distance between original embeddings and their "homogenized" Lie group versions during training.

Key Results and Visual Confirmation

The method was evaluated on standard benchmarks including ICEWS14, ICEWS05-15, and GDELT. The results consistently showed improvements across all baseline models without adding any extra trainable parameters.

  • Accuracy Gains: On the ICEWS05-15 dataset, the TeAST model's Mean Reciprocal Rank (MRR) improved from 48.8 to 59.2—a significant 10.4-point jump.
  • Efficiency: Training time overhead was minimal, increasing only slightly (e.g., from 30 to 32 minutes for TComplEx).
t-SNE visualizations of factor distributions across different models showing the alignment of entities, relations, and timestamps.

As shown in the t-SNE visualizations above, standard models (a, c) exhibit distinct, separated clusters for entities, relations, and timestamps. After applying the proposed Lie group mapping (b, d), the three distributions overlap almost perfectly, empirically confirming the reduction in heterogeneity.

Limitations and Future Directions

While highly effective, the approach currently faces the "cold start" problem, as it cannot predict relationships for entities not seen during training. Additionally, the matrix logarithm implementation requires specific mathematical criteria for the embedding rank (e.g., $\sqrt{2 \times rank}$ must be an integer), which may necessitate minor hyperparameter adjustments. Future research may explore other manifold types, such as those capturing cyclical or seasonal temporal patterns.

Summary updated

Abstract

Recent studies have highlighted the effectiveness of tensor decomposition methods in the Temporal Knowledge Graphs Embedding (TKGE) task. However, we found that inherent heterogeneity among factor tensors in tensor decomposition significantly hinders the tensor fusion process and further limits the performance of link prediction. To overcome this limitation, we introduce a novel method that maps factor tensors onto a unified smooth Lie group manifold to make the distribution of factor tensors approximating homogeneous in tensor decomposition. We provide the theoretical proof of our motivation that homogeneous tensors are more effective than heterogeneous tensors in tensor fusion and approximating the target for tensor decomposition based TKGE methods. The proposed method can be directly integrated into existing tensor decomposition based TKGE methods without introducing extra parameters. Extensive experiments demonstrate the effectiveness of our method in mitigating the heterogeneity and in enhancing the tensor decomposition based TKGE models.
BibTeX
@inproceedings{li2025mitigating,
  author = {Jiang Li and Xiangdong Su and Guanglai Gao},
  title = {{Mitigating Heterogeneity among Factor Tensors via Lie Group Manifolds for Tensor Decomposition Based Temporal Knowledge Graph Embedding}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2025},
  url = {https://openreview.net/forum?id=J679JBfC1N},
}
Figure