Introduction
Building conversational agents that feel "human" and "engaging" remains a significant challenge in natural language processing. While neural models have improved at next-utterance prediction, they often struggle with the nuances of multi-turn dialogue, frequently becoming repetitive or overly generic. This research investigates how fine-tuning low-level "knobs"—such as repetition, specificity, and question-asking—impacts human judgments of conversation quality.
Methodology
The researchers employed two primary techniques to inject control into neural text generation:
- Conditional Training (CT): Models are trained with discrete control variables (e.g., specificity levels), allowing users to specify desired attributes at test time.
- Weighted Decoding (WD): A test-time intervention that modifies the scoring function during beam search to reward or penalize specific features, such as rare words or repeated n-grams.
The study targeted four key attributes: repetition (n-gram overlap), specificity (normalized inverse document frequency), response-relatedness (sentence embedding similarity), and question-asking (presence of "?"). Models were pre-trained on 2.5 million Twitter pairs and fine-tuned on the PersonaChat dataset.
Key Insights: The "Goldilocks" Zone and Beyond
The study revealed several critical insights into what humans value in a digital conversation partner:
- Repetition is the "Silent Killer": Eliminating repetition is the single most effective baseline improvement. Simple n-gram blocking significantly boosted every human metric across the board.
- Optimal Specificity: There is a non-linear relationship between specificity and engagement. While generic responses are perceived as boring, excessive specificity can lead to nonsensical content or "hallucinations" that disrupt the flow.
- The Inquisitiveness Paradox: Humans preferred bots that asked questions approximately 65% of the time—more than double the rate found in human-to-human gold data (28.8%). Users seem to enjoy the engagement of being asked about themselves.
Results and Performance
By systematically optimizing these controllable attributes, the authors developed a model that matched the "engagingness" of state-of-the-art systems like the NeurIPS ConvAI2 winner ("Lost in Conversation"). Notably, this was achieved using 12x less training data, highlighting that strategic generation control can be as effective as massive scale.
In A/B testing, specificity-controlled models were preferred over baselines up to 84% of the time. Furthermore, increasing specificity and response-relatedness directly improved "listening" scores, suggesting that users perceive more detailed, on-topic responses as evidence of the bot's attention.
Limitations and Future Directions
A primary trade-off exists between "interestingness" and "making sense." High-control settings, particularly in weighted decoding, can sometimes produce ungrammatical or "degenerate" text. Additionally, while this study used static control settings, future work could explore dynamic control, where the model adjusts its attributes (e.g., becoming more inquisitive or specific) based on the real-time state and sentiment of the conversation.
Summary updated