This paper tackles the challenge of creating powerful, open-domain chatbots that can run efficiently on user devices, a crucial step for the next generation of digital assistants. The authors introduce a modular, lightweight framework that achieves comparable performance to massive models while drastically reducing parameter count, enabling on-device deployment.
TL;DR
This paper presents a modular, lightweight chatbot framework that significantly reduces model size (by 90%) while maintaining comparable performance to large, state-of-the-art models. This approach enables on-device deployment for open-domain chatbots, making them more practical for real-world applications like digital assistants.
What Problem Does This Paper Address?
The current trend in open-domain chatbots is to build massive, parameter-heavy models that achieve impressive conversational abilities through extensive pre-training. While effective, these models are too large and computationally expensive to run directly on user devices. This limitation hinders their deployment in scenarios requiring low latency and offline capabilities, such as on smartphones or smart speakers. The paper addresses this by aiming to create chatbots that are both knowledgeable and efficient, capable of handling diverse conversational tasks while fitting within resource-constrained environments.
Methods
The core innovation of this work is a modular, lightweight framework that breaks down the chatbot's functionality into distinct components, each optimized for a specific task. This contrasts with monolithic, end-to-end models. The proposed pipeline includes:
- Reference Resolution: This module ensures that follow-up questions in a conversation are self-contained by resolving anaphoric references and ellipses, improving the understanding of multi-turn dialogues.
- Factual Classifier: This component determines if a user's query requires factual information from external knowledge sources. This is crucial for efficient processing, allowing the system to offload complex factual queries to servers if necessary, while handling simpler queries on-device.
- Subjective Response Generation: This module handles chit-chat and opinion-based questions. It's a fine-tuned, 90M parameter Transformer model (based on Blender) specifically adapted for entertainment-domain conversations.
- ExtractNParaphrase: For factual questions, this module retrieves relevant passages, extracts answer spans, and paraphrases the information into a natural and engaging response.
- Inconsistency/Toxicity Predictor: This crucial component acts as a safeguard, detecting and mitigating inconsistent or toxic responses to ensure a coherent and safe conversational experience.
Datasets
The authors leverage a combination of existing and internal datasets for training and evaluation:
- Internal Media Dataset: A newly curated dataset of 100,000 movie-themed turns, designed to mimic natural human conversation with explicit instructions for anaphoric references. This dataset is used for various components, including training the factual classifier, reference resolution, and for human evaluation.
- QReCC (Question Rewriting and Contextual Conversation): Used for training the reference resolution, passage retrieval, and answer span extraction models.
- Wizard of Wikipedia (WoW): Provides a broad knowledge base for the subjective response generation model.
- ConvAI2: Used to train the subjective response generation model, focusing on engaging and persona-based conversations.
- Empathetic Dialogues (ED): Helps the subjective model learn to respond with appropriate tone and empathy.
- Blended Skill Talk (BST): A compilation of WoW, ConvAI2, and ED, used to train the subjective response generation model.
- DECODE: Used for training the inconsistency/toxicity predictor.
Key Insights and Ideas
- Modularity is Key for Low-Resource Adaptation: By breaking down the complex task of open-domain conversation into specialized modules, the authors demonstrate that it's possible to achieve high performance with significantly fewer parameters. This modular approach allows for targeted optimization of each component.
- Reference Resolution Boosts Multi-Turn Coherence: Explicitly addressing anaphoric references in multi-turn dialogues is shown to be critical for generating higher-quality, more coherent responses, a finding that aligns with previous work but is integrated here as a core module.
- Factual vs. Subjective Distinction is Practical: The Factual Classifier is a smart way to manage computational resources. It allows the system to intelligently route queries, ensuring factual accuracy when needed without burdening the entire model with deep knowledge retrieval for every turn.
- Balancing General Knowledge with Domain Specialization: The subjective response generation model, while based on a large pre-trained model (Blender), is fine-tuned on specific entertainment domains. This allows it to retain general conversational abilities while excelling in targeted areas.
Key Results
The paper reports significant achievements:
- 90% Parameter Reduction: The proposed LED framework achieves comparable performance to large models like Blender (90M and 2.7B parameters) while using drastically fewer parameters (186M and 276M for LED with and without reference resolution, respectively).
- Comparable Performance: The LED chatbot achieves comparable performance to state-of-the-art models on both automatic metrics (Perplexity) and human evaluations (Sensibleness and Specificity Average - SSA). For instance, on the internal media dataset, the LED 186M model achieves an SSA of 83.70, close to Blender 2.7B's 86.56.
- Improved Response Quality: The inclusion of the reference resolution component demonstrably improves the quality of generated responses in multi-turn conversations.
Limitations and Future Work
The authors acknowledge several limitations:
- Over-Agreement: The chatbot tends to agree with the user, potentially leading to less engaging conversations. This is attributed to the inclusion of the Empathetic Dialogues dataset, which incentivizes agreeable responses.
- Non-Neutral Persona on Sensitive Issues: The inconsistency/toxicity predictor occasionally fails, leading to non-neutral responses on controversial topics.
- Questionable Advice: The model sometimes provides unnecessary or questionable advice, particularly when drawing from datasets like Wizard of Wikipedia, which emphasizes displaying expert knowledge.
Future work includes:
- Improving Inconsistency/Toxicity Detection: Further enhancing the predictor to better handle sensitive issues.
- Unified Embedding Layer: Investigating a common embedding layer for all modules to further minimize parameter count without sacrificing performance.
- Leveraging User Feedback: Incorporating transactional request feedback signals to improve conversational task completion rates, especially for natural language understanding components.
Summary updated