Improving Neural Conversational Models with Entropy-Based Data Filtering

Richard Csaky, Patrik Purgai, Gábor RecskiACL 2019 · DOI

Abstract

Open-domain Neural Conversational Models (NCMs) frequently generate generic, unengaging responses like "I don't know." This paper proposes an unsupervised, entropy-based data filtering method to address this "boringness" problem. By identifying and removing high-entropy utterances—those associated with a wide, unpredictable variety of responses—from training sets, standard Transformer models can be trained to produce more diverse and specific dialogue.

Introduction

Despite advancements in architecture, chatbots often suffer from a lack of specificity. This is driven by two phenomena: the one-to-many problem, where a single prompt has numerous valid responses leading the model to predict a generic "average," and the many-to-one problem, where generic responses are over-represented across diverse contexts. While previous solutions relied on expensive human-annotated priors (e.g., personas), this research explores whether the training data itself can be improved through unsupervised filtering.

Methodology: Entropy-Based Filtering

The authors introduce Entropy-Based Data Filtering to quantify the uncertainty of an utterance. If a source utterance is paired with many different responses in a dataset, it possesses high entropy, marking it as too open-ended for effective learning.

The researchers tested three representation methods:

  1. IDENTITY: Treating unique strings as distinct categories.
  2. AVG-EMBEDDING: Clustering via averaged word embeddings.
  3. SENT2VEC: Utilizing sophisticated sentence-level embeddings.

Utterances were clustered using the Mean Shift algorithm, and entropy was calculated for the resulting distributions. Filtering strategies focused on removing high-entropy source utterances (SOURCE), target utterances (TARGET), or both (BOTH).

Entropy and Utterance Length

A key observation during data analysis was the strong correlation between entropy and utterance length. High-entropy utterances, which the filtering process aims to remove, are predominantly short. Longer utterances tend to be more specific and thus have lower entropy.

Entropy vs. Utterance Length

The Overfitting Paradox

In traditional NLP, training typically halts when validation loss stops decreasing. However, the authors observed a "paradox" in dialogue modeling: evaluation metrics such as embedding similarity and BLEU continued to improve even after the cross-entropy loss indicated the model was overfitting. This suggests that standard loss functions are insufficient proxies for conversational quality.

Divergence between Loss and Evaluation Metrics

Results and Discussion

The proposed filtering was evaluated on the DailyDialog, Cornell Movie-Dialogs, and Twitter datasets using 17 metrics.

  • Target Filtering Effectiveness: Filtering generic responses (TARGET) proved more impactful than filtering open-ended prompts (SOURCE). Removing these "safe" replies forced the model to learn more specific mappings.
  • Diversity and Quality: Models trained on filtered data achieved higher "distinct-n" scores, indicating a richer vocabulary. Qualitative analysis showed that filtered models provided context-specific answers where baseline models defaulted to generic advice.
  • Simplicity Wins: Interestingly, the IDENTITY method often outperformed complex clustering. This suggests that the noise introduced by current sentence clustering techniques may outweigh the benefits of semantic grouping.

Conclusion

Entropy-based filtering offers a promising, unsupervised path toward more engaging chatbots. While the method significantly boosts response diversity, future work should incorporate human evaluation to confirm perceived quality and explore more robust sentence representations like BERT to improve clustering accuracy. Balancing diversity with coherence remains a critical challenge for future conversational AI research.

Summary updated

BibTeX
@inproceedings{csaky2019improving,
  author = {Richard Csaky and Patrik Purgai and Gábor Recski},
  title = {{Improving Neural Conversational Models with Entropy-Based Data Filtering}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2019},
  doi = {10.18653/v1/p19-1567},
}
Figure