Abstract
Building open-domain chatbots is a challenging area for machine learning research. While prior work has shown that scaling neural models in the number of parameters and the size of the data they are trained on gives improved results, we show that other ingredients are important for a high-performing chatbot. Good conversation requires a number of skills that an expert conversationalist blends in a seamless way: providing engaging talking points and listening to their partners, and displaying knowledge, empathy and personality appropriately, while maintaining a consistent persona. We show that large scale models can learn these skills when given appropriate training data and choice of generation strategy. We build variants of these recipes with 90M, 2.7B and 9.4B parameter models, and make our models and code publicly available. Human evaluations show our best models are superior to existing approaches in multi-turn dialogue in terms of engagingness and humanness measurements. We then discuss the limitations of this work by analyzing failure cases of our models.
Introduction
Building an "open-domain" chatbot—one capable of discussing any topic from music to history—remains a grand challenge in AI. Historically, the field has focused on scale: increasing parameters and training data. However, massive models often suffer from being "dull," repetitive, or lacking a consistent persona. This research explores the "recipes"—combinations of architectures, training data, and inference techniques—required to create a bot that feels truly human-like, engaging, and knowledgeable.
Methods and Training Data
The researchers explored three main model architectures based on the Transformer framework: a Retriever, a Generator, and a hybrid Retrieve and Refine (RetNRef) approach. While models were scaled up to 9.4 billion parameters, the core technical novelty lies in the training and decoding strategies.
The Blended Skill Talk (BST) Dataset
Models were first pre-trained on 1.5 billion Reddit comments. The critical step was fine-tuning on the Blended Skill Talk (BST) tasks, which combine three distinct datasets: ConvAI2 (persona consistency), Empathetic Dialogues (empathy), and Wizard of Wikipedia (factual knowledge). This blending allows the model to seamlessly switch between being a knowledgeable expert and a sympathetic listener.
Decoding Strategies
The study found that a model's "perplexity" (predictive accuracy) doesn't always correlate with human preference. To avoid dull responses, they implemented specific decoding strategies:
- Minimum Beam Length: Forcing longer, more descriptive answers to avoid short, generic responses.
- Subsequence Blocking: Preventing n-gram repetition to reduce "looping" behavior.
- Unlikelihood Training: Penalizing the model for over-using common, dull phrases.
Key Results and Evaluation
The authors used ACUTE-Eval, a pairwise human preference test, to compare their best model against existing state-of-the-art systems like Google’s Meena. In these tests, humans are shown two side-by-side conversations and asked which speaker they would prefer to talk to for a long conversation.
The 9.4B parameter BST model significantly outperformed Meena, winning 75% of the time on engagingness and 65% on humanness. A key insight was the "length bias": humans strongly prefer longer responses. By simply forcing the model to talk more via minimum beam length, human evaluation scores improved dramatically.
Limitations and Future Work
Despite these successes, several failure cases persist. The models still suffer from hallucination (stating false facts confidently), lack of long-term memory (forgetting context after 128 tokens), and self-contradiction. Furthermore, while the models can mimic conversation, they often lack a deep understanding of complex logic or humor. Future research will focus on architectures with longer memory and real-time external knowledge retrieval to mitigate these issues.
Summary updated