Abstract
Neural conversational models often struggle with the "dull response problem," producing generic replies like "I don't know." This is largely due to the "one-to-many" nature of dialogue and the limitations of Maximum Likelihood Estimation (MLE) on simple query-response pairs. In this paper, we propose Posterior-GAN, a novel generative adversarial network that leverages query-response-future turn triples. By using dual generative discriminators—one forward-looking and one backward-looking—the model encourages responses that are both informative and contextually coherent. Experimental results on DailyDialog and OpenSubtitles datasets demonstrate significant improvements in both automatic and human evaluations.
The Challenge of Dialogue: The Dull Response Problem
Standard neural conversational models are typically trained on pairs of queries ($x$) and responses ($y$). However, human dialogue is inherently "loosely coupled"; a single query can lead to a vast array of valid responses. When models are trained using MLE, they tend to "average" these possibilities, resulting in safe but uninformative responses. Furthermore, these models often ignore the "future" of the conversation, failing to generate responses that naturally bridge the current context with subsequent turns.
Methods: Learning from the Future
To address these issues, Posterior-GAN shifts the training unit from a pair $(x, y)$ to a triple $(x, y, z)$, where $z$ represents the future turn. The framework consists of a generator and two specialized generative discriminators.
- The Generator ($G_\theta$): An attention-based Seq2Seq model that generates a response $y$ from query $x$.
- Forward Generative Discriminator ($D_{\phi1}$): Instead of a binary classifier, this is a generative model that attempts to predict the actual future turn $z$ given the generated response $y$. A response that provides a strong signal for the future receives a higher reward.
- Backward Generative Discriminator ($D_{\phi2}$): This component works in reverse, predicting the response $y$ given the future turn $z$. This ensures the response acts as a coherent bridge between the past and the future.
The generator is optimized via Policy Gradient reinforcement learning. By using generative discriminators (calculating cross-entropy) instead of binary ones, the model avoids "reward saturation" and receives more granular feedback on semantic coherence.
Experimental Results
The model was evaluated on the DailyDialog and OpenSubtitles (OSDb) datasets. Posterior-GAN consistently outperformed baselines like standard Seq2Seq-att, Adver-REGS, and DP-GAN.
Qualitative Analysis
As shown in the examples below, Posterior-GAN produces responses that are noticeably more specific and contextually relevant than its predecessors. By training with the future turn in mind, the model learns to provide information that facilitates the next stage of the conversation.
Quantitative and Human Evaluation
- Diversity: On DailyDialog, Posterior-GAN achieved significantly higher "Distinct" scores, indicating a reduction in generic responses.
- Semantic Similarity: It led in BLEU and embedding-based metrics, showing its outputs align more closely with human-generated text.
- Human Judgment: In blind tests, human annotators ranked Posterior-GAN as the most coherent and informative model. On DailyDialog, it achieved a mean rank of 3.22 for informativeness, significantly better than the 3.89 achieved by the baseline.
Conclusion
Posterior-GAN demonstrates that "looking ahead" is a powerful strategy for dialogue generation. By treating the future turn as a supervisor and employing dual generative discriminators, the model bridges the gap between past context and future possibilities. While training with reinforcement learning remains complex, the resulting improvements in response quality mark a significant step toward more natural and engaging AI conversation.
Summary updated