Neural Response Ranking for Social Conversation: A Data-Efficient Approach

Igor Shalyminov, Ondřej Dušek, Oliver LemonEMNLP 2018

TL;DR

This paper introduces a neural network-based response ranker for social chatbots that can be trained effectively using only dialogue length as a supervision signal, eliminating the need for costly and sparse user ratings. This data-efficient approach significantly simplifies and reduces the expense of collecting training data for engaging conversational AI.

What Problem Does This Paper Address?

Building truly engaging and natural social chatbots is a significant challenge. Current approaches often rely on user-provided ratings to train response selection models, which are expensive, time-consuming to collect, and often sparse. This paper addresses the problem of efficiently training these models by exploring alternative, more readily available supervision signals, specifically focusing on dialogue length as a proxy for user engagement. The goal is to make the development of high-quality social conversational agents more accessible.

Methods

The core of the paper is a neural ranker architecture. This model takes a dialogue context (the last few turns) and a candidate response as input. It uses a shared RNN encoder (specifically GRU cells) to create latent representations of both the context and the response. These representations are then concatenated with additional dialogue features (like sentiment, timestamp, and named entities) and fed into a feed-forward neural network (MLP), the "Predictor," to output a score between 0 and 1, indicating the likelihood of a good response.

The novelty lies in the training methodology. While traditional methods use explicit user ratings, this paper proposes training the ranker using either:

  1. User ratings: A standard approach, used as a baseline.
  2. Dialogue length: A novel approach where the model is trained to predict the length of the conversation. The intuition is that longer conversations are generally more engaging.

The model is trained using Mean Squared Error (MSE) loss.

Datasets

The primary dataset used is from the 2017 Alexa Prize challenge. This dataset consists of over 200,000 real user-chatbot dialogues.

  • Training Data: Two datasets were sampled from the Alexa Prize data:
  • Rating Dataset: Contains context-response pairs from highly rated dialogues (ratings 4-5) as positive instances and low-rated dialogues (ratings 1-2) as negative instances.
  • Length Dataset: Contains context-response pairs from long dialogues (target value > 0.7) as positive instances and short dialogues (target value < 0.3) as negative instances. Both datasets contain 500,000 instances.

  • Evaluation Data: A separate User Feedback dataset was created, containing 24,982 (context, good_response, bad_response) tuples. This dataset was curated using explicit user feedback phrases (e.g., "great, thank you") and sentiment analysis, aiming to provide an evaluation signal independent of the training objectives.

Key Insights and Ideas

  • Dialogue length as a proxy for engagement: The paper strongly suggests that dialogue length, even when approximated, is a surprisingly effective signal for training a social chatbot ranker. This is a significant finding because it bypasses the need for expensive human annotations.
  • Data efficiency is paramount: The research highlights the critical need for data-efficient methods in dialogue systems, given the high cost of data collection and annotation.
  • Neural ranker superiority: The proposed neural ranker consistently outperforms baseline methods (handcrafted and linear rankers) across different training signals.
  • Length-based training can surpass rating-based training: With sufficient data, training solely on dialogue length can lead to better performance than training on sparse user ratings.

Key Results

  • The neural ranker trained on user ratings achieved a Precision@1 of 0.847 on the evaluation dataset.
  • The neural ranker trained on dialogue length achieved a Precision@1 of 0.824 on the evaluation dataset with a medium-sized training set.
  • When trained on larger amounts of data (up to 1,000,000 data points), the length-based neural ranker reached a Precision@1 of 0.87, outperforming the rating-based version.
  • The neural ranker significantly outperformed handcrafted and linear rankers, with a much larger gap between the neural ranker and its next-best alternatives.
  • The dual-encoder ranker, while a strong architecture, was found to be less efficient for this specific problem.

Limitations and Future Work

  • Limitations:
    • The paper acknowledges that dialogue length is an approximation and might not always perfectly correlate with user satisfaction (e.g., long dialogues could be due to misunderstandings).
  • The user feedback dataset for evaluation, while aiming for independence, still relies on heuristics and specific phrases.
  • The dual-encoder ranker's performance was limited by computational constraints on extended datasets.
  • Future Work:
    • Investigate adversarial training methods to further improve the model.
  • Explore pairwise ranking approaches (e.g., following Burges et al., 2005) instead of the current pointwise approach.
  • Employ contextual sampling of negative responses for more efficient pairwise training.
  • Continue to explore other readily available supervision signals beyond ratings and length.

Summary updated

BibTeX
@inproceedings{shalyminov2018neural,
  author = {Igor Shalyminov and Ondřej Dušek and Oliver Lemon},
  title = {{Neural Response Ranking for Social Conversation: A Data-Efficient Approach}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2018},
}
Figure