Abstract
Keeping the conversation consistent and avoiding its repetition are two key factors to construct an intelligent multi-turn knowledge-grounded dialogue system. Although some works tend to combine history with external knowledge such as personal background information to boost dialogue quality, they are prone to ignore the fact that incorporating the same knowledge multiple times into the conversation leads to repetition. The main reason is the lack of effective control over the use of knowledge on the conversation level. So we design a history-adaption knowledge incorporation mechanism to build an effective multi-turn dialogue model. Our proposed model addresses repetition by recurrently updating the knowledge from the conversation level and progressively incorporating it into the history step-by-step. And the knowledge-grounded history representation also enhances the conversation consistency. Experimental results show that our proposed model significantly outperforms several retrieval-based models on some benchmark datasets. The human evaluation demonstrates that our model can maintain conversation consistent and reduce conversation repetition.
Introduction
In the development of intelligent chatbots, "knowledge-grounded" systems represent the state-of-the-art by utilizing external facts—such as user personas or factual documents—to enrich dialogue. However, two persistent challenges hinder their effectiveness: repetition and contextual decay. Current models often lack a mechanism to track which information has already been shared, leading to redundant responses. Furthermore, in long conversations, models struggle to maintain a coherent thread as the relevance of external knowledge shifts turn-by-turn. This paper proposes a history-adaption knowledge incorporation mechanism that treats knowledge as a dynamic resource, updating its representation turn-by-turn to ensure dialogue remains fresh and contextually relevant.
Methodology
The proposed architecture consists of three primary layers designed to manage the flow of information across multiple turns:
- Encoder Layer: Utilizes Bidirectional Gated Recurrent Units (Bi-GRU) and self-attention to encode external knowledge, conversation history, and response candidates into vector representations.
- History-Adaption Layer: This core innovation employs a gating mechanism to control information flow. It performs Knowledge Updating, using a "forget gate" to prioritize unused information for future turns, and creates a Knowledge-Aware History by integrating facts directly into the conversation's memory.
- Hierarchical History Encoder: A high-level GRU aggregates individual turns into a comprehensive context vector for response selection.
Evaluation and Results
The model was evaluated on two benchmark datasets: Persona-Chat, focusing on character consistency, and CMUDoG, involving document-grounded movie discussions.
As shown in the performance comparison, the proposed model consistently outperforms state-of-the-art retrieval-based baselines. On the CMUDoG dataset, the model achieved a significant jump in hits@1 from 60.3 to 82.7, demonstrating its superior ability to handle complex, document-heavy knowledge compared to standard Transformers.
Human evaluations further confirmed these findings, with the model scoring significantly higher for consistency and reduced repetition (2.36 vs. 2.07 on a 3-point scale).
Conclusion
This work demonstrates that external knowledge in dialogue systems should not be static. By implementing a history-adaption mechanism with a recurrent update gate, the model effectively tracks conversational progress and avoids redundancy. Future work will explore applying this logic to generative models and improving the mining of "hard negative" candidates to further refine retrieval accuracy.
Summary updated