Relevant and Informative Response Generation using Pointwise Mutual Information

Junya Takayama, Yuki AraseACL 2019

TL;DR

This paper tackles the problem of generic responses in neural dialogue systems by introducing a novel method that leverages Positive Pointwise Mutual Information (PPMI) to identify and promote the generation of relevant and informative keywords. By modifying the loss function to reward the inclusion of these keywords, the model achieves more diverse and content-rich responses compared to existing approaches.

What Problem Does This Paper Address?

Modern neural dialogue systems, often built using sequence-to-sequence (Seq2Seq) models, excel at generating fluent and grammatically correct responses. However, they frequently fall into the trap of producing generic, uninformative replies like "Yes" or "I don't know." This happens because standard Seq2Seq training penalizes any deviation from the reference response equally, leading the model to favor high-frequency, safe words. While some prior work, like MMI-bidi, has attempted to increase response diversity, they often overlook the crucial aspect of informativeness, leading to responses that might be diverse but not necessarily relevant or engaging for the user. This paper aims to bridge this gap by generating responses that are both diverse and informative, enhancing user experience.

Methods

The core innovation of this paper lies in its "keyword handler" and modified loss function.

  1. Keyword Identification (PPMI): The model first identifies keywords that are likely to co-occur between an utterance and its response. It does this by calculating Positive Pointwise Mutual Information (PPMI) between words in utterance-response pairs from a large corpus. PPMI quantifies how much more likely two words are to appear together than if they were independent. The top-k words with the highest PPMI scores for a given utterance are selected as keywords.
  2. Keyword-Guided Decoding: The identified keywords are then used to influence the response generation process.
  3. Keyword Scores: A vector of keyword scores for all words in the decoder's vocabulary is computed.
  4. Decoder Input: This keyword score vector is processed by a Multi-Layer Perceptron (MLP) and concatenated with the encoder's output to form the initial state of the decoder. This primes the decoder to consider keyword relevance.
  5. Loss Function Modification: The standard cross-entropy loss is augmented with a new loss term that rewards the generation of identified keywords. This term, denoted as $l_v$, is designed to decrease as keywords are generated. To handle the discrete nature of word generation and enable backpropagation, the paper employs the Gumbel-Softmax trick for differentiable pseudo-sampling of decoder outputs. The final loss function is a weighted sum of the cross-entropy loss and the keyword-based loss.

Datasets

The authors used two datasets for training and evaluation:

  • OpenSubtitles: A large, open-domain corpus of movie subtitles. For this work, consecutive subtitle lines were treated as utterance-response pairs, resulting in approximately 2 million pairs.
  • Twitter: A Japanese conversational corpus crawled from Twitter, using "@" mentions to identify replies. This dataset yielded about 1.3 million utterance-response pairs.

For both datasets, 10k pairs were set aside for validation and another 10k for testing, with the remainder used for training.

Key Insights and Ideas

  • PPMI as a Signal for Relevance and Informativeness: The paper demonstrates that PPMI is an effective metric for identifying words that are not just relevant to an utterance but also likely to contribute to an informative response, going beyond simple co-occurrence.
  • Gumbel-Softmax for Differentiable Sampling: The use of Gumbel-Softmax is crucial for enabling gradient-based optimization of a loss function that relies on sampling discrete words, a common challenge in sequence generation.
  • Balancing Relevance and Diversity: The proposed model, by explicitly rewarding keyword generation, strikes a better balance between generating relevant content and diversifying responses compared to previous methods that might achieve diversity at the cost of relevance.
  • The Trade-off Between Informativeness and Repetition: While the model successfully generates more informative responses, it also exhibits a tendency towards repetition, suggesting that simply promoting keywords might not be enough to prevent redundant outputs.

Key Results

The proposed model, both on its own ("Proposed") and when combined with MMI-bidi ("Proposed + MMI"), generally outperformed the baseline Seq2Seq and MMI-bidi models across several evaluation metrics on both OpenSubtitles and Twitter datasets.

  • Improved Metrics: The model showed improvements in BLEU, NIST, dist (diversity), and ent (entropy) scores. Notably, the "Proposed" model achieved a higher NIST score than Seq2Seq on the Twitter dataset, indicating better relevance for less frequent n-grams.
  • Longer Responses: The proposed models tended to generate longer responses than the baseline Seq2Seq.
  • Increased Repetition: A drawback observed was a higher repetition rate in the generated responses, particularly for the "Proposed + MMI" model.

Limitations and Future Work

The authors acknowledge a key limitation: the tendency for their model to over-generate repetitive words. They attribute this to the time-invariant nature of the keyword scores.

Future work directions include:

  • Updating Keyword Scores: Developing a mechanism to update keyword scores dynamically to avoid repetition.
  • Human Evaluation: Conducting human evaluations to complement automatic metrics and gain deeper insights into response quality.
  • Qualitative Analysis: Performing more in-depth qualitative analysis of generated responses.
  • Hyperparameter Tuning: Investigating the effect of the hyperparameter $\alpha$ (which balances cross-entropy and keyword loss) on overall performance.
  • Suppressing Over-generation: Developing specific mechanisms to suppress repetitive outputs.

Summary updated

BibTeX
@inproceedings{takayama2019relevant,
  author = {Junya Takayama and Yuki Arase},
  title = {{Relevant and Informative Response Generation using Pointwise Mutual Information}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2019},
}
Figure