Responsive and Self-Expressive Dialogue Generation

Kozo Chikai, Junya Takayama, Yuki AraseACL 2019

TL;DR

This paper introduces a novel dialogue generation model that aims to overcome the "generic response" problem in chatbots by simultaneously incorporating domain-awareness and sentiment-richness. By drawing inspiration from the interpersonal process model of intimacy, their model generates more engaging and human-like responses, significantly outperforming traditional sequence-to-sequence models in human evaluations.

What Problem Does This Paper Address?

Current dialogue systems, especially those for casual chit-chat, often suffer from generating overly generic responses. This lack of engagement can make interactions feel robotic and uninteresting, hindering user experience. While previous research has explored ways to make responses more engaging by reacting to user topics or embodying emotions, this paper tackles the challenge of achieving both "conversational responsiveness" (showing concern for what was said) and "self-expression" (sharing thoughts and feelings) simultaneously in a single model. They propose that these two factors, crucial for human intimacy, are also key to creating more engaging AI dialogue agents.

Methods

The core of their approach is a modified sequence-to-sequence (SEQ2SEQ) architecture that integrates two key components:

  • Domain-Awareness: The encoder is enhanced to predict the domain of the user's utterance. This is achieved by using TWEET2VEC, a pre-trained model known for its effectiveness in understanding short, colloquial text, to predict domain labels. This domain information is then explicitly fed to the decoder.
  • Sentiment-Richness: The decoder is designed to embody sentiment polarity. It takes a desired sentiment label (positive, negative, or neutral) as input and uses an "external memory" mechanism to control the generation of responses, switching between sentiment-specific words and generic words. This allows the model to express subjective feelings.

The model is trained using a two-stage process:

  1. Pre-training: Sub-models (SEQ2SEQ, TWEET2VEC, and sentiment embeddings) are trained independently on large general and in-domain corpora. This helps overcome data scarcity for sentiment-rich dialogues.
  2. Fine-tuning: The pre-trained sub-models are connected and fine-tuned on the in-domain corpus to integrate domain labels and input utterances.

Datasets

  • General Corpus: Approximately 1.6 million utterance-response pairs were collected from conversational tweets and the NTCIR Short Text Conversation Task.
  • In-domain Corpus: Approximately 29k baseball-related and 28k game-related conversations were crawled from public Facebook Groups.
  • Sentiment Lexicon: A custom sentiment lexicon was constructed by extending existing lexicons with casual and recent sentiment words from Twitter.

Key Insights and Ideas

  • Interpersonal Process Model as Inspiration: The paper effectively translates human intimacy factors (responsiveness and self-expression) into concrete computational mechanisms for dialogue generation.
  • Simultaneous Achievement: The key innovation is the simultaneous integration of domain-awareness and sentiment-richness, rather than focusing on one aspect alone.
  • TWEET2VEC for Domain Prediction: Leveraging TWEET2VEC proved effective for domain prediction on colloquial text, a common challenge in dialogue systems.
  • External Memory for Sentiment Control: The external memory mechanism provides a novel way to explicitly control the sentiment polarity of generated responses.
  • Pre-training for Data Scarcity: The two-stage training approach is crucial for handling the limited availability of domain-specific and sentiment-rich dialogue data.

Key Results

  • Human Evaluation: The proposed model significantly outperformed a standard SEQ2SEQ model in human evaluations.
  • Domain-awareness: 89% of the model's responses were deemed domain-aware, compared to only 10.9% for SEQ2SEQ.
  • Sentiment-richness: 72% of the model's responses showed salient sentiments, versus 28.2% for SEQ2SEQ.
  • Overall: 68.1% of the model's responses were judged as both domain-aware and sentiment-rich, compared to a mere 2.7% for SEQ2SEQ.
  • Fluency and Consistency: While the proposed model achieved slightly lower scores in fluency (95.5% vs. 99.5%) and consistency (75.3% vs. 77.3%), the authors attribute this to the model generating longer, more specific responses, which is a trade-off for increased engagement.
  • Automatic Evaluation: The model achieved a higher BLEU score (1.54) than SEQ2SEQ (1.39), though the authors note BLEU's limitations in correlating with human judgment for dialogue systems.

Limitations and Future Work

  • Sentiment Nuance: The paper acknowledges that the sentiment annotation process is simple and that neutral responses can be misjudged. Future work aims to improve polarity strength and resolve context-dependent sentiment cases, potentially using self-attention.
  • Domain Coverage: The current study focuses on two specific domains (baseball and Pokémon Go). The authors suggest extending the dataset to cover a wider range of domains.
  • Longer Conversations: The current study focuses on single-turn conversations. Future work could explore extending the model to handle multi-turn dialogues.
  • External Memory Complexity: While effective, the external memory mechanism adds complexity. Further research could explore more efficient implementations.

Summary updated

BibTeX
@inproceedings{chikai2019responsive,
  author = {Kozo Chikai and Junya Takayama and Yuki Arase},
  title = {{Responsive and Self-Expressive Dialogue Generation}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2019},
}
Figure