Learning to Select Knowledge for Response Generation in Dialog Systems

Rongzhong Lian, Min Xie, Fan Wang, Jinhua Peng, Hua WuIJCAI 2019 · DOI

TL;DR

This paper tackles the problem of generating informative responses in dialogue systems by proposing a novel knowledge selection mechanism. Their approach leverages both prior (utterance-based) and posterior (utterance-and-response-based) knowledge distributions to ensure more appropriate knowledge is selected during training, leading to better response generation.

What Problem Does This Paper Address?

End-to-end neural dialogue systems, while capable of generating fluent responses, often fall into the trap of producing generic or uninformative replies (e.g., "I don't know"). To combat this, researchers have explored incorporating external knowledge, such as structured data or text. However, a critical challenge remains: how to effectively select the right piece of knowledge from a potentially large collection to inform the response. Previous methods often relied on semantic similarity between the utterance and knowledge (a "prior" distribution), but this can be insufficient as multiple pieces of knowledge might seem relevant. The authors highlight that a more effective approach would consider the actual knowledge used in a true response (a "posterior" distribution), but this is unavailable during inference. This discrepancy between what's available during training (posterior) and inference (prior) hinders the model's ability to learn proper knowledge selection.

Methods

The core innovation is a neural model that explicitly separates and then aligns the prior and posterior knowledge distributions.

  • Prior Knowledge Module: This module estimates a probability distribution over knowledge based solely on the input utterance. It uses an attention mechanism to measure the relevance between the utterance and each knowledge item. This is the knowledge selection mechanism available during inference.
  • Posterior Knowledge Module: This module estimates a probability distribution over knowledge by considering both the input utterance and the ground truth response. This provides a more accurate signal of which knowledge was actually used.
  • Knowledge Manager: This component orchestrates the prior and posterior modules. During training, it aims to minimize the KL divergence between the prior and posterior distributions. This forces the model to learn to approximate the posterior using only the prior.
  • Knowledge Selection: During training, knowledge is sampled based on the posterior distribution. During inference, it's sampled based on the prior distribution.
  • Decoder: The selected knowledge is then fed into a decoder (either a standard GRU with concatenated inputs or a Hierarchical Gated Fusion Unit for a "softer" integration) to generate the response.

The overall loss function combines three components: KL divergence loss (for knowledge selection alignment), Negative Log-Likelihood (NLL) loss (for word order), and Bag-of-Words (BOW) loss (for ensuring knowledge relevance to the response).

Datasets

The paper evaluates their model on two publicly available datasets:

  • Persona-chat: This dataset consists of conversations where each participant is assigned a persona profile, which serves as the knowledge.
  • Wizard-of-Wikipedia: This dataset features conversations where one agent (the "wizard") has access to knowledge from Wikipedia.

The authors note that they did not use the ground-truth knowledge provided in Wizard-of-Wikipedia, focusing instead on scenarios where this knowledge is unknown, aligning with their proposed mechanism.

Key Insights and Ideas

  • The Prior-Posterior Discrepancy is Key: The paper clearly articulates and quantifies the challenge posed by the difference between knowledge selection signals available during training (posterior) and inference (prior). This is a fundamental problem in knowledge-grounded dialogue.
  • Aligning Distributions for Better Learning: By explicitly training the model to approximate the posterior distribution using the prior, they enable effective knowledge selection even when only the prior is available at inference time.
  • Posterior as a "Ground Truth" for Selection: The posterior distribution, derived from the actual response, acts as a more reliable teacher for what constitutes "appropriate" knowledge compared to just utterance-based relevance.
  • Soft vs. Hard Knowledge Integration: The comparison between their "PostKS(concat)" (hard) and "PostKS(fusion)" (soft) models suggests that a more flexible, "soft" integration of knowledge into the decoder leads to slightly better performance.

Key Results

The proposed "PostKS" models (both concat and fusion variants) significantly outperform baseline models across various metrics on both datasets.

  • Automatic Evaluation: PostKS models achieved higher BLEU, Distinct, and Knowledge R/P/F1 scores. For instance, on Persona-chat, PostKS(fusion) achieved a Knowledge F1 of 0.0870, significantly higher than MemNet(soft)'s 0.0223.
  • Human Evaluation: Human evaluators rated the responses from PostKS models as more natural, relevant, and informative, with PostKS(fusion) consistently scoring highest (e.g., 0.97 on Persona-chat, 1.02 on Wizard-of-Wikipedia Test Seen).
  • Case Studies: Qualitative examples show that their models generate more specific and engaging responses, even posing follow-up questions, compared to baselines that produce bland or irrelevant replies.
  • Transferability: Applying their knowledge selection mechanism to a strong Transformer model (LIC) also led to substantial improvements in knowledge R/P/F1.

Limitations and Future Work

  • Limitations: The paper acknowledges that the posterior distribution is unknown during inference, which is the core problem they address. While their method mitigates this, it's an inherent challenge. The "hard" decoder variant might be less flexible than the "soft" one.
  • Future Work: The authors plan to extend their knowledge selection mechanism to handle multi-turn conversations, suggesting that the current work focuses primarily on single-turn knowledge selection.

Summary updated

BibTeX
@inproceedings{lian2019learning,
  author = {Rongzhong Lian and Min Xie and Fan Wang and Jinhua Peng and Hua Wu},
  title = {{Learning to Select Knowledge for Response Generation in Dialog Systems}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2019},
  doi = {10.24963/ijcai.2019/706},
}
Figure