Abstract
Data-driven, knowledge-grounded neural conversation models are capable of generating more informative responses. However, these models have not yet demonstrated that they can zero-shot adapt to updated, unseen knowledge graphs. This paper proposes a new task about how to apply dynamic knowledge graphs in neural conversation model and presents a novel TV series conversation corpus (DyKgChat) for the task. Our new task and corpus aids in understanding the influence of dynamic knowledge graphs on responses generation. Also, we propose a preliminary model that selects an output from two networks at each time step: a sequence-to-sequence model (Seq2Seq) and a multi-hop reasoning model, in order to support dynamic knowledge graphs. To benchmark this new task and evaluate the capability of adaptation, we introduce several evaluation metrics and the experiments show that our proposed approach outperforms previous knowledge-grounded conversation models. The proposed corpus and model can motivate the future research directions.
Introduction
Most knowledge-grounded dialogue models rely on static facts, but the real world is dynamic—relationships evolve and new information emerges constantly. Current neural conversation models are typically trained on a fixed snapshot of knowledge. If a relationship in the graph changes—for example, if two characters in a show go from being friends to enemies—traditional models often fail to update their responses unless they are painstakingly retrained on the new data.
This paper identifies a critical gap in "zero-shot adaptation": the ability of a model to immediately respect a changed fact in a knowledge graph (KG) at inference time without retraining. To solve this, the authors introduce a new task where models must generate responses based on evolving KGs.
The DyKgChat Corpus
To support this research, the authors introduced the DyKgChat corpus, which uses scripts from two popular TV series to capture complex, shifting human relationships:
- HGZHZ (Hou Gong Zhen Huan Zhuan): A Chinese palace drama known for intricate alliances (76 episodes).
- Friends: The classic English sitcom (236 episodes).
The researchers manually constructed KGs for these shows, capturing relations like LoverOf, EnemyOf, and ResidenceOf. Crucially, they sampled "dynamic" versions of these graphs to test how models handle changes in facts between different scenes or episodes. Data analysis revealed that most knowledge-driven responses in these shows require 2 to 3 hops of reasoning (e.g., identifying "the residence of my friend"), highlighting the need for sophisticated graph navigation.
Methodology: The Qadpt Model
The authors propose Qadpt (Quick Adaptive Dynamic Knowledge-Grounded Neural Conversation Model). Unlike previous models that use static graph embeddings—which effectively "bake" knowledge into the model weights—Qadpt separates the reasoning logic from the graph structure.
Key components of the architecture include:
- Seq2Seq Backbone: A standard GRU-based encoder-decoder for natural language processing.
- The Controller: A mechanism that decides at each time step whether to generate a "generic" word (like "the" or "hello") or a "knowledge" word from the KG.
- Multi-hop Reasoning Model: This core innovation constructs a transition matrix based on the current KG's adjacency. It uses a Markov chain process to "walk" N-hops through the graph to find the correct entity. Because the transition matrix is built directly from the KG at runtime, any change in the graph (adding or deleting an edge) immediately changes the model's output distribution without requiring any retraining.
Key Results and Insights
- Zero-Shot Superiority: Qadpt significantly outperformed baselines like MemNet and KAware in the "Accurate Change Rate" metric. When the KG was modified, Qadpt was far more likely to produce the new correct entity rather than hallucinating the old one.
- Knowledge Accuracy: On the HGZHZ dataset, Qadpt achieved a Knowledge Word Accuracy (KW-Acc) of 57.61%, nearly five times higher than a standard Seq2Seq model (12.10%).
- Explicit Reasoning vs. Embeddings: The study demonstrates that explicit multi-hop reasoning is superior to graph embeddings for dynamic tasks. While embeddings are difficult to update, a transition matrix can be updated instantly by modifying the adjacency table.
Conclusion
DyKgChat provides a challenging benchmark for dynamic knowledge grounding. While Qadpt shows strong performance, the authors note a data imbalance in the Friends dataset, where only 16% of turns contain KG entities compared to 59% in HGZHZ. Future work will focus on scaling to larger open-domain KGs and improving the controller's ability to transition between natural language and structured data.
Summary updated