Abstract
Open-domain Neural Conversational Models (NCMs) frequently generate generic, unengaging responses like "I don't know." This paper proposes an unsupervised, entropy-based data filtering method to address this "boringness" problem. By identifying and removing high-entropy utterances—those associated with a wide, unpredictable variety of responses—from training sets, standard Transformer models can be trained to produce more diverse and specific dialogue.
Introduction
Despite advancements in architecture, chatbots often suffer from a lack of specificity. This is driven by two phenomena: the one-to-many problem, where a single prompt has numerous valid responses leading the model to predict a generic "average," and the many-to-one problem, where generic responses are over-represented across diverse contexts. While previous solutions relied on expensive human-annotated priors (e.g., personas), this research explores whether the training data itself can be improved through unsupervised filtering.
Methodology: Entropy-Based Filtering
The authors introduce Entropy-Based Data Filtering to quantify the uncertainty of an utterance. If a source utterance is paired with many different responses in a dataset, it possesses high entropy, marking it as too open-ended for effective learning.
The researchers tested three representation methods:
- IDENTITY: Treating unique strings as distinct categories.
- AVG-EMBEDDING: Clustering via averaged word embeddings.
- SENT2VEC: Utilizing sophisticated sentence-level embeddings.
Utterances were clustered using the Mean Shift algorithm, and entropy was calculated for the resulting distributions. Filtering strategies focused on removing high-entropy source utterances (SOURCE), target utterances (TARGET), or both (BOTH).
Entropy and Utterance Length
A key observation during data analysis was the strong correlation between entropy and utterance length. High-entropy utterances, which the filtering process aims to remove, are predominantly short. Longer utterances tend to be more specific and thus have lower entropy.
The Overfitting Paradox
In traditional NLP, training typically halts when validation loss stops decreasing. However, the authors observed a "paradox" in dialogue modeling: evaluation metrics such as embedding similarity and BLEU continued to improve even after the cross-entropy loss indicated the model was overfitting. This suggests that standard loss functions are insufficient proxies for conversational quality.
Results and Discussion
The proposed filtering was evaluated on the DailyDialog, Cornell Movie-Dialogs, and Twitter datasets using 17 metrics.
- Target Filtering Effectiveness: Filtering generic responses (TARGET) proved more impactful than filtering open-ended prompts (SOURCE). Removing these "safe" replies forced the model to learn more specific mappings.
- Diversity and Quality: Models trained on filtered data achieved higher "distinct-n" scores, indicating a richer vocabulary. Qualitative analysis showed that filtered models provided context-specific answers where baseline models defaulted to generic advice.
- Simplicity Wins: Interestingly, the IDENTITY method often outperformed complex clustering. This suggests that the noise introduced by current sentence clustering techniques may outweigh the benefits of semantic grouping.
Conclusion
Entropy-based filtering offers a promising, unsupervised path toward more engaging chatbots. While the method significantly boosts response diversity, future work should incorporate human evaluation to confirm perceived quality and explore more robust sentence representations like BERT to improve clustering accuracy. Balancing diversity with coherence remains a critical challenge for future conversational AI research.
Summary updated