PVGRU: Generating Diverse and Relevant Dialogue Responses via Pseudo-Variational Mechanism

YongKang Liu, Shi Feng, Daling Wang, Yifei Zhang, Hinrich SchuetzeOpenReview 2022 · OpenReview

Abstract

We investigate response generation for multi-turn dialogue in generative-based chatbots. Existing generative models based on RNNs (Recurrent Neural Networks) usually employ the last hidden state to summarize the sequences, which makes models unable to capture the subtle variability observed in different dialogues and cannot distinguish the differences between dialogues that are similar in composition. In this paper, we propose a Pseudo-Variational Gated Recurrent Unit (PVGRU) component without posterior knowledge through introducing a recurrent summarizing variable into the GRU, which can aggregate the accumulated distribution variations of subsequences. PVGRU can perceive the subtle semantic variability through summarizing variables that are optimized by the devised distribution consistency and reconstruction objectives. In addition, we build a Pseudo-Variational Hierarchical Dialogue (PVHD) model based on PVGRU. Experimental results demonstrate that PVGRU can broadly improve the diversity and relevance of responses on two benchmark datasets.

Introduction

Developing conversational agents that generate diverse and contextually relevant responses remains a significant challenge in Natural Language Processing. Generative models must navigate complex "one-to-many" mappings, where a single prompt can elicit numerous valid responses, and "many-to-one" mappings, where different contexts converge on similar replies. Standard Recurrent Neural Networks (RNNs) often struggle with these dynamics, tending toward generic, "safe" responses like "I don't know."

While Variational Autoencoders (VAEs) were introduced to model this inherent uncertainty, they are frequently hampered by posterior collapse, where the model learns to ignore latent variables, and a training-inference inconsistency caused by the use of target responses during training that are unavailable during deployment.

Capturing Subtle Semantic Variability

One of the primary motivations for the PVGRU is the need to distinguish between dialogues that appear similar but possess distinct semantic nuances.

Subtle semantic variability between dialogue contexts and the model's perception of these differences.

As shown in Figure 1, minor changes in dialogue context (e.g., the number of people mentioned) should lead to significantly different internal representations and subsequent responses. The PVGRU is designed to perceive these subtle shifts through its unique summarizing mechanism.

Methodology: The Pseudo-Variational GRU (PVGRU)

The core contribution of this work is the Pseudo-Variational Gated Recurrent Unit (PVGRU). Unlike a standard GRU that relies solely on a hidden state, the PVGRU introduces a recurrent summarizing variable ($v$) to aggregate distribution variations over time.

The internal architecture of the PVGRU cell (a) and the hierarchical PVHD model structure (b).

Key Mechanisms

  • Pseudo-Variational Mechanism: By avoiding a posterior distribution that depends on the target response, the PVGRU eliminates the training-inference gap and inherently prevents posterior collapse.
  • Consistency Objective: This loss function ensures that the change in the model's internal state at each step is statistically consistent with the input data.
  • Reconstruction Objective: To ensure the summarizing variable effectively captures the context, the model is tasked with reconstructing the preceding sequence from this variable.

These components are integrated into the Pseudo-Variational Hierarchical Dialogue (PVHD) model, which applies the PVGRU at both the word level (Encoder) and the utterance level (Context) to manage the hierarchical flow of conversation.

Experimental Results

The PVHD model was evaluated on two benchmarks: DailyDialog (daily life conversations) and DSTC7-AVSD (social media dialogues about video clips).

Diversity and Relevance

Experimental results show that PVHD significantly outperforms traditional seq2seq, HRED, and advanced variational models like CVAE and VHCR.

  • Diversity: The model achieved substantial gains in "Distinct-1" and "Distinct-2" metrics, improving Dist-1 by over 11% on DailyDialog compared to several baselines.
  • Relevance: Higher BLEU and Rouge-L scores indicate that the generated responses are more contextually grounded and relevant than those of competing models.
  • Human Evaluation: Blind human assessments confirmed that PVHD produces more diverse and relevant responses than other RNN-based architectures.

Conclusion and Future Work

The PVGRU component successfully addresses the limitations of standard RNNs and VAEs in dialogue generation by capturing subtle semantic variability without the risk of posterior collapse. While the current implementation is RNN-based, future research will focus on adapting this pseudo-variational logic for Transformer architectures and further closing the fluency gap with large-scale pre-trained models.

Summary updated

Abstract

We investigate response generation for multiturn dialogue in generative chatbots. Existing generative models based on RNNs (Recurrent Neural Networks) usually employ the last hidden state to summarize the history, which makes models unable to capture the subtle variability observed in different dialogues and cannot distinguish the differences between dialogues that are similar in composition. In this paper, we propose Pseudo-Variational Gated Recurrent Unit (PVGRU). The key novelty of PVGRU is a recurrent summarizing variable that aggregates the accumulated distribution variations of subsequences. We train PVGRU without relying on posterior knowledge, thus avoiding the training-inference inconsistency problem. PVGRU can perceive subtle semantic variability through summarizing variables that are optimized by two objectives we employ for training: distribution consistency and reconstruction. In addition, we build a Pseudo-Variational Hierarchical Dialogue (PVHD) model based on PVGRU. Experimental results demonstrate that PVGRU can broadly improve the diversity and relevance of responses on two benchmark datasets.
BibTeX
@misc{liu2022pvgru,
  author = {YongKang Liu and Shi Feng and Daling Wang and Yifei Zhang and Hinrich Schuetze},
  title = {{PVGRU: Generating Diverse and Relevant Dialogue Responses via Pseudo-Variational Mechanism}},
  year = {2022},
  eprint = {2212.09086},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2212.09086},
  howpublished = {OpenReview},
}
Figure