Posterior-GAN: Towards Informative and Coherent Response Generation with Posterior Generative Adversarial Network

Shaoxiong Feng, Hongshen Chen, Kan Li, Dawei Yin

Abstract

Neural conversational models often struggle with the "dull response problem," producing generic replies like "I don't know." This is largely due to the "one-to-many" nature of dialogue and the limitations of Maximum Likelihood Estimation (MLE) on simple query-response pairs. In this paper, we propose Posterior-GAN, a novel generative adversarial network that leverages query-response-future turn triples. By using dual generative discriminators—one forward-looking and one backward-looking—the model encourages responses that are both informative and contextually coherent. Experimental results on DailyDialog and OpenSubtitles datasets demonstrate significant improvements in both automatic and human evaluations.

The Challenge of Dialogue: The Dull Response Problem

Standard neural conversational models are typically trained on pairs of queries ($x$) and responses ($y$). However, human dialogue is inherently "loosely coupled"; a single query can lead to a vast array of valid responses. When models are trained using MLE, they tend to "average" these possibilities, resulting in safe but uninformative responses. Furthermore, these models often ignore the "future" of the conversation, failing to generate responses that naturally bridge the current context with subsequent turns.

Methods: Learning from the Future

To address these issues, Posterior-GAN shifts the training unit from a pair $(x, y)$ to a triple $(x, y, z)$, where $z$ represents the future turn. The framework consists of a generator and two specialized generative discriminators.

The Posterior-GAN architecture models the relationship between the query (x), generated response (y), and future turn (z) using dual generative discriminators.
  1. The Generator ($G_\theta$): An attention-based Seq2Seq model that generates a response $y$ from query $x$.
  2. Forward Generative Discriminator ($D_{\phi1}$): Instead of a binary classifier, this is a generative model that attempts to predict the actual future turn $z$ given the generated response $y$. A response that provides a strong signal for the future receives a higher reward.
  3. Backward Generative Discriminator ($D_{\phi2}$): This component works in reverse, predicting the response $y$ given the future turn $z$. This ensures the response acts as a coherent bridge between the past and the future.

The generator is optimized via Policy Gradient reinforcement learning. By using generative discriminators (calculating cross-entropy) instead of binary ones, the model avoids "reward saturation" and receives more granular feedback on semantic coherence.

Experimental Results

The model was evaluated on the DailyDialog and OpenSubtitles (OSDb) datasets. Posterior-GAN consistently outperformed baselines like standard Seq2Seq-att, Adver-REGS, and DP-GAN.

Qualitative Analysis

As shown in the examples below, Posterior-GAN produces responses that are noticeably more specific and contextually relevant than its predecessors. By training with the future turn in mind, the model learns to provide information that facilitates the next stage of the conversation.

Qualitative comparison of generated responses. Posterior-GAN avoids the repetitive or generic outputs seen in baseline models by considering the conversational future.

Quantitative and Human Evaluation

  • Diversity: On DailyDialog, Posterior-GAN achieved significantly higher "Distinct" scores, indicating a reduction in generic responses.
  • Semantic Similarity: It led in BLEU and embedding-based metrics, showing its outputs align more closely with human-generated text.
  • Human Judgment: In blind tests, human annotators ranked Posterior-GAN as the most coherent and informative model. On DailyDialog, it achieved a mean rank of 3.22 for informativeness, significantly better than the 3.89 achieved by the baseline.

Conclusion

Posterior-GAN demonstrates that "looking ahead" is a powerful strategy for dialogue generation. By treating the future turn as a supervisor and employing dual generative discriminators, the model bridges the gap between past context and future possibilities. While training with reinforcement learning remains complex, the resulting improvements in response quality mark a significant step toward more natural and engaging AI conversation.

Summary updated

Abstract

Neural conversational models learn to generate responses by taking into account the dialog history. These models are typically optimized over the query-response pairs with a maximum likelihood estimation objective. However, the query-response tuples are naturally loosely coupled, and there exist multiple responses that can respond to a given query, which leads the conversational model learning burdensome. Besides, the general dull response problem is even worsened when the model is confronted with meaningless response training instances. Intuitively, a high-quality response not only responds to the given query but also links up to the future conversations, in this paper, we leverage the query-response-future turn triples to induce the generated responses that consider both the given context and the future conversations. To facilitate the modeling of these triples, we further propose a novel encoder-decoder based generative adversarial learning framework, Posterior Generative Adversarial Network (Posterior-GAN), which consists of a forward and a backward generative discriminator to cooperatively encourage the generated response to be informative and coherent by two complementary assessment perspectives. Experimental results demonstrate that our method effectively boosts the informativeness and coherence of the generated response on both automatic and human evaluation, which verifies the advantages of considering two assessment perspectives.
BibTeX
@inproceedings{feng2020posterior,
  author = {Shaoxiong Feng and Hongshen Chen and Kan Li and Dawei Yin},
  title = {{Posterior-GAN: Towards Informative and Coherent Response Generation with Posterior Generative Adversarial Network}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2020},
  url = {https://openreview.net/forum?id=H-VbcaPotzc},
}
Figure