Abstract
Abigail See, Stephen Roller, Douwe Kiela, Jason Weston. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers).
2019.
This paper moves beyond treating "conversation quality" as a black box by identifying and controlling the specific low-level attributes—like repetition, specificity, and question-asking—that drive human judgments. By using Conditional Training and Weighted Decoding to fine-tune these behaviors, the authors developed a model that matches the performance of state-of-the-art dialogue agents while using 12x less training data.
Introduction
Despite the proliferation of neural dialogue models, they often suffer from "boring bot" syndrome: producing generic, repetitive, or entirely unrelated responses. While researchers typically evaluate these models using a single "overall quality" score, we lack a granular understanding of which specific conversational behaviors actually lead to a "good" conversation.
Most existing work focuses on "next-utterance prediction," which ignores the flow of a multi-turn interaction. This paper addresses the gap between low-level model outputs and high-level human perceptions, seeking to answer: Which controllable attributes make a bot engaging, and how can we manipulate them to improve performance?
Deconstructing Conversation Quality
The authors move beyond treating "conversation quality" as a black box by identifying and controlling specific low-level attributes that drive human judgments. These attributes are mapped to conversational aspects and ultimately to overall quality metrics like humanness and engagingness.
Methods
The authors examine two primary methods for controlling four key attributes: repetition, specificity, response-relatedness, and question-asking.
- Conditional Training (CT): The model is trained on a control variable $z$ (e.g., a bucketed range of specificity). At test time, providing a specific $z$ tells the model what "style" to produce. This is effective for high-level attributes like question-asking.
- Weighted Decoding (WD): This technique modifies the model's word-selection process at test time only. By adding a weight to specific features (like rare words for specificity or n-gram blocking for repetition), the authors can force the model to change its behavior without retraining.
The study systematically compares these methods, finding that while CT produces more "natural" (in-distribution) text, WD is more powerful for enforcing strict constraints like avoiding repetition.
Key Insights and the "Goldilocks Zone"
- The Repetition Ceiling: Repetition is the single most destructive factor in human quality judgments. Simply fixing "external repetition" (repeating oneself across turns) provides a massive, immediate boost in perceived quality.
- Specificity vs. Making Sense: Increasing specificity (using rarer words) makes a bot more interesting, but it’s a risky trade-off. If a model is pushed to be too specific, it often starts producing nonsensical or "hallucinated" details. This creates a "Goldilocks Zone" where moderate specificity is preferred over extreme genericness or extreme specificity.
- The "Goldilocks Zone" for Questions: Humans prefer bots that ask questions, but there is a sweet spot. The most engaging models asked questions about 65% of the time—significantly higher than the human average in the dataset (~29%), but not so high that the bot seemed "nosy."
- Engagingness \neq Humanness: Interestingly, the most engaging bots weren't necessarily the ones rated as most "human." A bot can be a great, inquisitive conversationalist without successfully tricking a human into thinking it's a person.
Key Results
- Efficiency: Their best-controlled model matched the engagingness score (3.1/4.0) of the winning entry in the NeurIPS ConvAI2 competition. Crucially, the competition winner was trained on nearly 12 times more data (the massive GPT model), proving that fine-grained control can compensate for sheer scale.
- Repetition Control: Implementing n-gram blocking and unigram weights improved the "Avoiding Repetition" human score from ~2.1 to ~2.9 (on a 3-point scale).
- Specificity: Using Weighted Decoding for specificity (weight = 4) significantly increased "Interestingness" and "Listening" scores compared to the baseline.
Limitations and Future Work
- Response-Relatedness Failure: Surprisingly, explicitly controlling for how "related" a response was to the previous turn did not improve human judgments. The authors hypothesize that more "related" models took more risks with rare words, leading to more nonsensical outputs.
- Subjectivity of "Interesting": The authors found that "interestingness" is highly subjective and difficult for crowdworkers to calibrate, suggesting a need for better-defined evaluation metrics.
- Automatic Optimization: The control settings in this study were tuned manually. Future work could involve models that automatically learn the optimal "Goldilocks" settings for different conversational contexts.
Summary updated