SRM-LLM: Semantic Relationship Mining with LLMs for Temporal Knowledge Graph Extrapolation

Fu Zhang, Panfeng Zhang, Jingwei ChengEMNLP 2025

TL;DR

This paper introduces SRM-LLM, a novel approach that leverages Large Language Models (LLMs) to mine latent semantic relationships between relations in Temporal Knowledge Graphs (TKGs). By understanding these relationships, SRM-LLM significantly improves the accuracy of predicting future events in TKGs, achieving state-of-the-art results.

What Problem Does This Paper Address?

Temporal Knowledge Graphs (TKGs) represent facts that evolve over time, making them crucial for understanding dynamic real-world phenomena like event prediction. A key challenge in TKGs is extrapolation: predicting future facts based on historical data. Existing methods often struggle because they overlook the complex, nuanced relationships between different types of relations (e.g., a causal link between "protest" and "sanctions"). This leads to incomplete representations of relations and hinders accurate reasoning. SRM-LLM aims to bridge this gap by using LLMs to uncover these hidden relational connections, thereby improving TKG extrapolation.

Methods

SRM-LLM introduces two main novel components:

  • Relational Semantic Association Module (RSA): This module uses LLMs to analyze the types of semantic associations between relations within a TKG. It identifies categories like "causal," "synonymous," and "oppositional" relationships. Based on these identified associations, it constructs "relational association subgraphs" which are then used to learn richer relation representations. This approach avoids the issue of isolated relation nodes that plague previous graph-based methods.
  • Rule-Based Historical Relation Retrieval Module (RHR): This module leverages temporal logical rules (inspired by TLogic) to extract globally relevant historical facts related to a query. These extracted facts are used to build a "global graph" that helps learn entity representations at a broader level, complementing the local, timestamp-specific entity representations.

The model then fuses these global and local entity representations for the final prediction.

Datasets

The authors evaluated SRM-LLM on five widely-used TKG datasets:

  • GDELT
  • WIKI
  • ICEWS14
  • ICEWS18
  • ICEWS05-15

They also created a "Sparse ICEWS14" dataset to test performance under data scarcity.

Key Insights and Ideas

  • LLMs are powerful for understanding relational semantics: The paper demonstrates that LLMs can effectively identify subtle semantic connections between relations, which are crucial for accurate TKG reasoning. This goes beyond simple structural connections.
  • Beyond "line graphs": Previous methods that transformed TKGs into line graphs often suffered from isolated relation nodes. SRM-LLM's relational association subgraph approach circumvents this, leading to more robust relation representations.
  • Global and Local Context is Key: Combining local, timestamp-specific entity evolution with global historical facts extracted via logical rules significantly boosts performance. The study suggests local history is slightly more impactful.
  • The number of relation types matters: While more relation types generally offer richer information, too many can lead to misclassification and increased complexity. The optimal number of identified association types is an important consideration.

Key Results

SRM-LLM achieved state-of-the-art (SOTA) results on multiple TKG datasets for entity prediction. Specifically:

  • On the ICEWS14 dataset, SRM-LLM improved the Hits@1 metric by 5.4% compared to baselines.
  • On the ICEWS05-15 dataset, it improved Hits@1 by 4.1%.
  • The model significantly outperformed static KG methods, highlighting the importance of temporal information.
  • It also showed superior performance compared to interpolation methods, demonstrating its strength in extrapolation.
  • In relation prediction, SRM-LLM also outperformed baseline methods.
  • Ablation studies confirmed the significant contributions of both the Relational Semantic Association Module and the Rule-Based Historical Relation Retrieval Module.

Limitations and Future Work

  • Complexity of Semantic Associations: The authors acknowledge that the five identified association types are a simplification, and real-world relations might exhibit more complex semantic connections. Future work could explore expanding and refining these types.
  • LLM Choice: The current work uses GPT-3.5-turbo. Future research could investigate the impact of different LLMs on the accuracy and efficiency of semantic association mining.
  • Dataset Sparsity: While SRM-LLM performs well on sparse datasets, further exploration into how LLMs can infer missing information in such scenarios is warranted.

Summary updated

BibTeX
@inproceedings{zhang2025srm,
  author = {Fu Zhang and Panfeng Zhang and Jingwei Cheng},
  title = {{SRM-LLM: Semantic Relationship Mining with LLMs for Temporal Knowledge Graph Extrapolation}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
}
Figure