What makes a good conversation? How controllable attributes affect human judgments

Abigail See, Stephen Roller, Douwe Kiela, Jason Weston

TL;DR

This paper investigates how specific, controllable attributes—like how often a bot asks questions or how rare its vocabulary is—impact human perceptions of conversational quality. By fine-tuning these "low-level" knobs using conditional training and weighted decoding, the authors developed a dialogue agent that matches the engagement levels of state-of-the-art models while using 12x less training data.

What Problem Does This Paper Address?

While neural dialogue models have become ubiquitous, they suffer from well-documented "personality" flaws: they are often repetitive, overly generic (the "I don't know" problem), and fail to maintain a natural flow over multiple turns.

The core challenge is that while we can measure "overall quality" through human Likert scales, we don't fully understand the underlying ingredients of a "good" conversation. Most research focuses on next-utterance prediction, but a single good sentence does not make a good conversation. The authors aim to bridge the gap between controllable algorithmic parameters and high-level human judgments like "engagingness," "interestingness," and "humanness."

Methods

The authors explore two primary methods for steering dialogue generation:

  1. Conditional Training (CT): The model is trained to predict the next word given the context and a control variable $z$ (e.g., a bucketed range of specificity). At test time, you simply feed the model the desired "knob setting" as an embedding concatenated to the decoder's input.
  2. Weighted Decoding (WD): This is an inference-time technique. The model’s raw output probabilities are modified by adding a bonus or penalty based on specific word features (e.g., penalizing words that appeared in previous turns to reduce repetition).

They apply these methods to four key attributes:

  • Repetition: Controlled via n-gram blocking and penalties.
  • Specificity: Measured using Normalized Inverse Document Frequency (NIDF), which quantifies word rareness.
  • Response-relatedness: Measured by the cosine similarity between the embeddings of the bot's response and the user's last input.
  • Question-asking: Controlling the frequency of interrogative tokens at the dialogue level (e.g., "aim for 70% questions").

Datasets

  • PersonaChat: The primary benchmark, consisting of multi-turn dialogues where each participant is assigned a specific "persona" (e.g., "I have two dogs").
  • Twitter: A massive dataset of 2.5 million message-response pairs used for pre-training the baseline model.
  • ConvAI2: The authors compare their results against the winning entry of this competition ("Lost in Conversation").

Key Insights and Ideas

  • Repetition is the "Silent Killer": The single most effective way to improve human ratings was simply to stop the bot from repeating itself. Reducing repetition improved scores across every metric, including fluency and listening.
  • The "Engagingness vs. Humanness" Gap: One of the most striking findings is that a bot can be highly engaging without being particularly human-like. The best models matched human levels of engagement but were still easily identified as bots.
  • Balance over Extremes: For specificity and question-asking, the relationship with quality is non-linear. Too much specificity leads to nonsensical "word salad," while too little leads to boring, generic responses. The "sweet spot" for question-asking (65.7%) was actually much higher than the human average (28.8%), suggesting that users enjoy the attention of a bot that asks about them.
  • Dialogue-Level vs. Utterance-Level Control: Instead of forcing every sentence to be a question, the authors controlled the distribution over the whole chat. This allows the model to decide when it makes sense to ask a question, leading to more natural flow.

Key Results

  • Efficiency: Their best model, which optimized repetition, specificity, and question-asking, achieved a raw engagingness score of 3.1/4.0. This matched the winner of the NeurIPS ConvAI2 competition, despite the competitor being trained on 12x more data (OpenAI GPT vs. a standard LSTM).
  • Specificity Win: In A/B tests, models with boosted specificity (using WD weight 4 or 6) were preferred by humans over 75% of the time compared to the baseline, primarily because they provided more information and felt more "interested."
  • Question-Asking: Increasing the question rate to ~66% significantly boosted engagingness, bringing the bot closer to human-level performance in that specific category.

Limitations and Future Work

  • Semantic Relatedness Failure: The authors found that Conditional Training (CT) struggled to learn the relationship between input and output for semantic relatedness. While Weighted Decoding (WD) could force relatedness, it often resulted in nonsensical lists of topically related words.
  • Subjectivity of "Interestingness": The authors noted that "interestingness" is highly subjective and hard to calibrate across different human annotators. They suggest that the PersonaChat task itself might have a natural "ceiling" for how interesting a conversation can actually be.
  • Automatic Optimization: A clear next step is to move away from manual "knob-turning" and develop algorithms that can automatically find the optimal balance of these attributes during a live conversation.

Summary updated

Abstract

A good conversation requires balance -- between simplicity and detail; staying on topic and changing it; asking questions and answering them. Although dialogue agents are commonly evaluated via human judgments of overall quality, the relationship between quality and these individual factors is less well-studied. In this work, we examine two controllable neural text generation methods, conditional training and weighted decoding, in order to control four important attributes for chitchat dialogue: repetition, specificity, response-relatedness and question-asking. We conduct a large-scale human evaluation to measure the effect of these control parameters on multi-turn interactive conversations on the PersonaChat task. We provide a detailed analysis of their relationship to high-level aspects of conversation, and show that by controlling combinations of these variables our models obtain clear improvements in human quality judgments.
BibTeX
@inproceedings{see2019what,
  author = {Abigail See and Stephen Roller and Douwe Kiela and Jason Weston},
  title = {{What makes a good conversation? How controllable attributes affect human judgments}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2019},
  url = {https://openreview.net/forum?id=SkbgaXWuZS},
}
Figure