TL;DR
This paper introduces Time-LowFER, a novel family of time-aware extensions to the LowFER model for temporal knowledge graph completion (TKGC). By proposing a cycle-aware time encoding scheme and exploring feature-based extensions like chained and factorized trilinear pooling, they achieve state-of-the-art results on TKGC benchmarks, demonstrating improved reasoning capabilities over existing methods.
What Problem Does This Paper Address?
Knowledge graphs (KGs) are powerful tools for organizing information, but static KGs miss crucial temporal context. Temporal Knowledge Graph Completion (TKGC) aims to fill this gap by reasoning over event and temporal KGs, predicting missing information that includes a time dimension. While tensor decomposition methods like LowFER have shown promise for static KGs, effectively incorporating time into these models remains a challenge. Existing approaches often struggle with representing time accurately, leading to inconsistencies and limitations in reasoning over time-sensitive data. This paper addresses the need for more robust and generalized methods for temporal reasoning in KGs.
Methods
The core contribution is the Time-LowFER framework, which extends the parameter-efficient LowFER model to handle temporal information. Key innovations include:
- Cycle-aware Time Encoding (CTE): Instead of treating timestamps as simple numerical values, they propose a multi-recurrent cycle-aware (MRCA) decomposition. This method breaks down time into various cyclical components (e.g., daily, weekly, monthly, yearly) to capture temporal patterns more effectively and create more generalized representations. This approach is model-agnostic.
- Feature-based Extensions: Beyond simple time modulation, they introduce two novel extensions that leverage feature interactions:
- Chained Factorized Bilinear Pooling (CFB): This method uses a multilinear approach by chaining bilinear models to capture interactions between entities, relations, and time.
- Factorized Trilinear Pooling (FTP): A more parameter-efficient specialization of CFB, it directly models the interactions between the three modalities.
- Time-LowFER Variants: The paper presents several Time-LowFER variants:
- LowFER-T: A basic time modulation extension.
- LowFER-TNT: A time-no-time modulation extension that combines static and dynamic relation features.
- LowFER-CFB: Incorporates the chained factorized bilinear pooling.
- LowFER-FTP: Incorporates the factorized trilinear pooling.
Datasets
The experiments were conducted on two widely used temporal knowledge graph datasets:
- ICEWS14: Contains facts from the year 2014, with daily temporal granularity.
- ICEWS05-15: Covers facts from 2005 to 2015, also with daily temporal granularity.
These datasets are subsets of the ICEWS (Integrated Crisis Early Warning System) database.
Key Insights and Ideas
- Time is Cyclical, Not Just Linear: The most significant insight is that treating time as a simple linear sequence is insufficient. Decomposing time into its inherent cyclical components (daily, weekly, yearly) leads to richer and more generalizable temporal representations.
- Feature-based Extensions Outperform Modulation: While time modulation is popular, the paper demonstrates that feature-based extensions (CFB and FTP) offer superior performance by more directly modeling the complex interactions between entities, relations, and time.
- Parameter Efficiency vs. Expressiveness Trade-off: The study highlights a trade-off between model expressiveness and computational cost. CFB achieves state-of-the-art results but is computationally more expensive than FTP, which offers a good balance.
- Model-Agnostic Time Encoding is Key: The proposed cycle-aware time encoding (CTE) is designed to be model-agnostic, meaning it can be readily integrated with various existing temporal KG embedding models, offering a flexible improvement.
Key Results
- State-of-the-Art Performance: The proposed Time-LowFER extensions, particularly LowFER-CFB, achieve state-of-the-art results on both ICEWS14 and ICEWS05-15 datasets across various metrics (MRR, H@10, H@3, H@1). For instance, LowFER-CFB achieves an MRR of 0.623 on ICEWS14 and 0.638 on ICEWS05-15, outperforming several strong baselines.
- Improved Reasoning: The TNT extension consistently shows improvements over the T extension, indicating the benefit of incorporating static relation information for frequently occurring relations.
- CTE Benefits: The CTE approach shows noticeable improvements in Hits@10 (e.g., 3.4% and 5.7% for modulation-based extensions on ICEWS datasets), suggesting its effectiveness in capturing temporal nuances for recall-oriented tasks.
Limitations and Future Work
- Overfitting Risk: The authors acknowledge that their current methods can overfit due to the lack of explicit regularization schemes like time-smoothness or nuclear 3-norm, which are present in some baselines.
- Computational Cost: While CFB offers superior performance, its computational cost is higher than other variants.
- Limited Performance Gains with CTE: Despite its conceptual advantages, the MRCA-based CTE shows limited performance gains and adds a computational footprint.
- Future Work: The authors suggest extending their work with regularization schemes. They also implicitly point towards exploring more efficient yet expressive feature-based extensions and further optimizing the time encoding methods.
Summary updated