Time-aware Multiway Adaptive Fusion Network for Temporal Knowledge Graph Question Answering

Yonghao Liu, Di Liang, Fang Fang, Sirui Wang, Wei Wu, Rui Jiang

Abstract

Knowledge graphs (KGs) have received increasing attention due to its wide applications on natural language processing. However, its use case on temporal question answering (QA) has not been well-explored. Most of existing methods are developed based on pre-trained language models, which might not be capable to learn temporal-specific presentations of entities in terms of temporal KGQA task. To alleviate this problem, we propose a novel Time-aware Multiway Adaptive (TMA) fusion network. Inspired by the step-by-step reasoning behavior of humans. For each given question, TMA first extracts the relevant concepts from the KG, and then feeds them into a multiway adaptive module to produce a temporal-specific representation of the question. This representation can be incorporated with the pre-trained KG embedding to generate the final prediction. Empirical results verify that the proposed model achieves better performance than the state-of-the-art models in the benchmark dataset. Notably, the Hits@1 and Hits@10 results of TMA on the CronQuestions dataset's complex questions are absolutely improved by 24% and 10% compared to the best-performing baseline. Furthermore, we also show that TMA employing an adaptive fusion mechanism can provide interpretability by analyzing the proportion of information in question representations.

Introduction

Knowledge Graphs (KGs) are essential for organizing information into triples of (Subject, Predicate, Object). However, real-world facts are dynamic; for instance, political roles and sports captaincies change over time. Temporal Knowledge Graph Question Answering (Temporal KGQA) addresses queries with specific time constraints, such as "Who was the captain of Brazil before Dani Alves?"

While standard Large Language Models (LLMs) excel at entity identification, they often struggle with temporal nuances. Researchers from Tsinghua University and Meituan Inc. have introduced the Time-aware Multiway Adaptive (TMA) fusion network, a framework designed to explicitly integrate temporal facts into the reasoning process to overcome this "temporal blindness."

Methodology

The TMA framework follows a "select-match-fusion-predict" paradigm to bridge the gap between textual questions and temporal knowledge.

The TMA framework overview, illustrating the SPO Selector, Multiway Matching, and Adaptive Fusion modules.
  1. SPO Selector: Acts as a retriever to identify the top 10 most relevant "Subject-Predicate-Object" (SPO) triples from the KG related to the entities in the question.
  2. Multiway Matching: Uses three distinct attention mechanisms—Concat, Dot, and Minus attention—to capture nuanced semantic relationships between the question and retrieved facts from different mathematical perspectives.
  3. Adaptive Fusion: Employs a gating mechanism to adaptively decide how much temporal information from the SPOs should be integrated into the final question embedding.
  4. Answer Prediction: Utilizes TComplEx embeddings to score potential entity or timestamp answers in a complex vector space.

Experimental Results

The researchers evaluated TMA on CronQuestions, the largest benchmark for temporal KGQA, involving over 400,000 question-answer pairs.

Main experimental results comparing TMA against state-of-the-art baselines on the CronQuestions dataset.

TMA established a new state-of-the-art performance with an overall Hits@1 of 0.784, significantly outperforming the previous best model, CronKGQA (0.647). The most dramatic improvements occurred in complex reasoning categories (e.g., "Before/After"), where TMA improved Hits@1 by 24% and Hits@10 by 10% over the best-performing baseline. Ablation studies confirmed that the SPO Selector and the adaptive fusion mechanism are the primary drivers of this increased accuracy.

Conclusion

TMA demonstrates that explicitly retrieving and fusing temporal triples significantly enhances the reasoning capabilities of models on temporal tasks. By calculating discrepancies through "Minus Attention" and weighting information via adaptive gating, the model provides both high performance and a degree of interpretability. Future work may focus on dynamic SPO selection and scaling these fusion techniques to larger generative models.

Summary updated

Abstract

Knowledge graphs (KGs) have received increasing attention due to its wide applications on natural language processing. However, its use case on temporal question answering (QA) has not been well-explored. Most of existing methods are developed based on pre-trained language models, which might not be capable to learn \emph{temporal-specific} presentations of entities in terms of temporal KGQA task. To alleviate this problem, we propose a novel \textbf{T}ime-aware \textbf{M}ultiway \textbf{A}daptive (\textbf{TMA}) fusion network. Inspired by the step-by-step reasoning behavior of humans. For each given question, TMA first extracts the relevant concepts from the KG, and then feeds them into a multiway adaptive module to produce a \emph{temporal-specific} representation of the question. This representation can be incorporated with the pre-trained KG embedding to generate the final prediction. Empirical results verify that the proposed model achieves better performance than the state-of-the-art models in the benchmark dataset. Notably, the Hits@1 and Hits@10 results of TMA on the CronQuestions dataset's complex questions are absolutely improved by 24\% and 10\% compared to the best-performing baseline. Furthermore, we also show that TMA employing an adaptive fusion mechanism can provide interpretability by analyzing the proportion of information in question representations.
BibTeX
@inproceedings{liu2023time,
  author = {Yonghao Liu and Di Liang and Fang Fang and Sirui Wang and Wei Wu and Rui Jiang},
  title = {{Time-aware Multiway Adaptive Fusion Network for Temporal Knowledge Graph Question Answering}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2023},
  url = {https://openreview.net/forum?id=aNeZZo3Iov},
}
Figure