Improving Response Quality with Backward Reasoning in Open-domain Dialogue Systems

Ziming Li, Julia Kiseleva, Maarten de RijkeSIGIR 2021 · DOI · OpenReview

Introduction

Standard open-domain dialogue systems often suffer from a "safe response" bias. Because these models are typically trained to maximize the likelihood of a response given a conversation history, they tend to favor high-frequency, generic phrases such as "I’m not sure" or "That’s cool." While statistically probable, these responses are uninformative and lead to dull, unengaging user experiences.

The "safe response" problem: generic replies (Response A) are statistically likely but less informative than specific ones (Response B).

To address this, researchers have developed Mirror, a dialogue generation model that leverages "backward reasoning." The core hypothesis is that a truly informative response should contain enough "topical glue" to allow a listener to reconstruct the original query. By enforcing this bidirectional coherence, the system is incentivized to generate more contextually relevant content without requiring external knowledge bases or complex reinforcement learning.

Methodology

The Mirror model shifts from simple forward generation to a bidirectional reasoning objective. The dialogue history is decomposed into the older context ($c$) and the immediate query ($x$), with the goal of generating a reply ($y$).

Instead of optimizing only for $P(y | c, x)$, Mirror aims to maximize the joint probability $P(x, y | c)$. This requires the model to simultaneously learn forward generation ($x \to y$) and backward reasoning ($y \to x$). This is achieved through a shared latent variable ($z$) that acts as a bridge in a shared latent space. During training, a variational lower bound (ELBO) ensures that $z$ captures sufficient information to satisfy both tasks. The architecture utilizes a context encoder, an utterance encoder, and four independent decoders to manage the various probability distributions required for this bidirectional objective.

Experimental Setup

The model was evaluated on two major dialogue benchmarks:

  • MovieTriples: Film transcripts consisting of three-turn dialogues (~166k training samples).
  • DailyDialog: A high-quality dataset of multi-turn daily communications (~65k training samples).

Results and Discussion

Mirror was compared against several baselines, including Seq2Seq, HRED, and MMI, using human evaluators on Amazon Mechanical Turk.

Head-to-head human evaluation results for Mirror against various baselines on MovieTriple and DailyDialog datasets.

As shown in the evaluation metrics, Mirror consistently outperformed or tied with more complex models. On the DailyDialog dataset, Mirror achieved a 50% win rate against the standard Seq2Seq model, with only a 26% loss rate. Qualitatively, Mirror demonstrates a superior grasp of situational context. For instance, where baselines might provide a generic "No," Mirror generates specific responses like "Well, I'm sure they'll have to wait," indicating a deeper integration of the dialogue history.

Conclusion

Mirror offers a robust method for improving dialogue quality by integrating bidirectional constraints directly into the training loss. While the simultaneous training of four decoders increases computational cost during the training phase, the model requires no additional overhead during inference. Future work may explore the application of this "invertibility" principle to task-oriented dialogues and complex question-answering systems, while also addressing potential risks to syntactic correctness when backward reasoning dominates the training process.

Summary updated

Abstract

Being able to generate informative and coherent dialogue responses is crucial when designing human-like open-domain dialogue systems. Encoder-decoder-based dialogue models tend to produce generic and dull responses during the decoding step because the most predictable response is likely to be a non-informative response instead of the most suitable one. To alleviate this problem, we propose to train the generation model in a bidirectional manner by adding a backward reasoning step to the vanilla encoder-decoder training. The proposed backward reasoning step pushes the model to produce more informative and coherent content because the forward generation step's output is used to infer the dialogue context in the backward direction. The advantage of our method is that the forward generation and backward reasoning steps are trained simultaneously through the use of a latent variable to facilitate bidirectional optimization. Our method can improve response quality without introducing side information (e.g., a pre-trained topic model). The proposed bidirectional response generation method achieves state-of-the-art performance for response quality.
BibTeX
@inproceedings{li2021improving,
  author = {Ziming Li and Julia Kiseleva and Maarten de Rijke},
  title = {{Improving Response Quality with Backward Reasoning in Open-domain Dialogue Systems}},
  booktitle = {International ACM SIGIR Conference on Research and Development in Information Retrieval},
  year = {2021},
  doi = {10.1145/3404835.3463004},
}
Figure