TL;DR
This paper tackles the problem of open-domain dialogue systems asking redundant questions, a common issue that degrades user experience in long conversations. The authors propose novel methods for both training and decoding to mitigate this, alongside a new dataset (NRQ) to facilitate research in this area. Their approach significantly reduces redundant questions, leading to more coherent and engaging chatbot interactions.
What Problem Does This Paper Address?
Modern neural dialogue systems, while impressive at generating short, coherent responses, struggle with maintaining quality in longer conversations. A key failure mode is the generation of redundant questions – questions whose answers have already been provided by the user or were asked previously. This is particularly problematic because it forces users to repeat information, leading to frustration and a poor user experience. Existing models are often trained on short conversations, prioritizing local turn-by-turn appropriateness over global conversational coherence, which exacerbates this issue. This paper aims to specifically address and reduce the rate of these redundant questions in open-domain dialogue systems.
Methods
The authors propose a multi-pronged approach to tackle redundant questions, combining training-time and decoding-time strategies:
- Unlikelihood Training: This method extends standard maximum likelihood estimation by introducing an "unlikelihood" objective. This objective actively penalizes the generation of undesirable responses (redundant questions) by pushing down their probability. They create negative training samples by pairing a dialogue context and a positive response with a modified partner persona that makes the response redundant.
- Contrastive Decoding: Adapting existing contrastive decoding techniques, they modify it for encoder-decoder models. This method aims to reduce repetition by favoring less likely but non-repetitive candidates during the decoding process, specifically by considering the similarity between candidate tokens and the dialogue context.
- Classification: They build a binary classifier to detect redundant questions. This classifier takes the dialogue context, partner persona, and generated response as input. They fine-tune pre-trained language models (XLNet, RoBERTa, DeBERTa) for this task.
- Combined Approach: The most effective strategy involves combining unlikelihood training with the redundant question classifier. This leverages the strengths of both methods: unlikelihood training addresses the underlying generation issue, while classification acts as a post-hoc filter.
Datasets
- NRQ Dataset (Non-Redundant Questions): The authors introduce a novel dataset specifically for this task. It consists of positive training samples (non-redundant questions) and negative samples (redundant questions). The positive samples are gathered from existing dialogue datasets (Wizard of Wikipedia, Empathetic Dialogues, Blended Skill Talk, Multi-Session Chat, Wizard of Internet), and negative samples are generated by creating contradictory partner personas. The dataset was filtered and split into training, validation, and testing sets.
- Existing Dialogue Datasets: Used for sourcing positive training samples for the NRQ dataset.
Key Insights and Ideas
- Persona Manipulation for Negative Sampling: A clever insight is the use of partner personas to create negative training examples. By slightly altering or adding to a persona, a previously non-redundant question can be made redundant, providing a direct signal for the model to learn from.
- Redundancy is a Global Coherence Problem: The paper highlights that redundant questions stem from a lack of global coherence, where models prioritize immediate turn relevance over the entire conversation history and context.
- Combined Methods Outperform Individual Ones: The most significant improvements come from combining different mitigation strategies, particularly unlikelihood training with a redundant question classifier. This suggests that a multi-faceted approach is necessary to effectively combat this issue.
- Bias in Training Data: The BB2 baseline's tendency to ask common, redundant questions is directly linked to the prevalence of those same questions in its training data, illustrating the impact of data bias on model behavior.
Key Results
- Reduced Redundant Question Rate: The proposed methods significantly reduce the redundant question rate. For the BB2 Baseline, the rate drops from 27.2% to 8.7% when using the combined unlikelihood training and classification approach.
- Improved Classification Accuracy: The redundant question classifier achieves high accuracy, with RoBERTa reaching 88.6% on the test set.
- Conversation Length Impact: The redundant question rate increases substantially with conversation length, highlighting the importance of long-term coherence. For example, the rate for BB2 Baseline increases from 18.4% at 30 turns to over 26.5% at 65 turns.
- Human Evaluation Wins: In human evaluations, the proposed approach significantly outperformed the original BB2 model in terms of overall question-asking ability.
Limitations and Future Work
- Resource Intensity: The proposed methods, especially those involving persona manipulation, increase inference time and GPU memory usage, making them less resource-friendly for prolonged conversations. Future work could explore using retrieval-based methods to select relevant personas more efficiently.
- High Redundant Rate Still Exists: Despite significant reductions, the redundant question rate remains at 8.7%. This is attributed to the model's difficulty in recognizing indirect relationships between personas and redundant questions.
- Scaling the Dataset: The authors suggest that scaling the NRQ dataset and employing better data augmentation techniques could further improve the model's ability to handle more complex cases of redundancy.
- Indirect Relations: The model still struggles with spotting redundancy when the connection to the persona is indirect (e.g., "I'm still in high school" implying "I don't have a job"). This suggests a need for models that can better infer implicit relationships.
Summary updated