Incorporating Casual Analysis into Diversified and Logical Response Generation

Jiayi Liu, Wei Wei, Zhixuan Chu, Xing Gao, Ji Zhang, Tan Yan, Yulin Kang

Abstract

Although the Conditional Variational Auto-Encoder (CVAE) model can generate more diversified responses than the traditional Seq2Seq model, the responses often have low relevance with the input words or are illogical with the question. A causal analysis is carried out to study the reasons behind, and a methodology of searching for the mediators and mitigating the confounding bias in dialogues is provided. Specifically, we propose to predict the mediators to preserve relevant information and auto-regressively incorporate the mediators into generating process. Besides, a dynamic topic graph guided conditional variational auto-encoder (TGG-CVAE) model is utilized to complement the semantic space and reduce the confounding bias in responses. Extensive experiments demonstrate that the proposed model is able to generate both relevant and informative responses, and outperforms the state-of-the-art in terms of automatic metrics and human evaluations.

Introduction

Open-domain dialogue systems face a classic "one-to-many" challenge: a single prompt can have infinite valid responses. While standard Seq2Seq models often produce generic, "boring" replies, Conditional Variational Auto-Encoders (CVAEs) improve diversity at the cost of relevance. CVAEs frequently pick up on "spurious statistical cues," leading to illogical details. This paper identifies this issue as a confounding bias in the causal relationship between the prompt and the response.

Methodology: Causal Analysis and SLARM

The authors propose SLARM (Sentence Level Auto-Regressive Model), which structures response generation through causal analysis. The process is divided into two steps:

  1. The Mediator Predictor: The model first predicts "direct responding semantics" ($y_{direct}$), which acts as a mediator ($m$). This mediator captures the core relevance of the prompt ($x$).
  2. The Auto-Regressive Response Generator (TGG-CVAE): Once the mediator is established, the model generates "supplementary semantics" ($y_{sup}$). This is achieved using a Topic Graph Guided CVAE, which uses a dynamic topic graph to ground supplementary information in logically related keywords.
Causal analysis framework for dialogue generation, illustrating the role of the mediator and confounding bias.

By separating responses into homogeneous direct answers and diversified supplementary info, SLARM mitigates the confounding bias ($c$) introduced by random sampling in traditional CVAEs.

Experiments and Results

The model was evaluated on the Short-Text Conversation (STC) dataset, containing 4.4 million post-comment pairs from Weibo.

Quantitative Evaluation

SLARM was compared against baselines including standard CVAE and MMPMS. As shown in the table below, SLARM achieved significant improvements in BLEU scores, indicating higher relevance and word overlap with ground truth. Ablation studies (w/o TGG, w/o CVAE) confirm the importance of the topic graph and the variational component.

Automatic evaluation results comparing SLARM against baselines and ablation studies.

Qualitative Evaluation

Human evaluations on Relevance, Informativeness, and Fluency placed SLARM as the top-performing model with an average score of 2.02. Case studies showed that SLARM generates logical extensions rather than nonsensical phrases common in other diverse models.

Conclusion

SLARM effectively addresses the "boring chatbot" problem by incorporating causal analysis into the generation process. By identifying mediators and using topic graphs to guide latent variables, it produces responses that are both diverse and logically consistent. Future work will focus on more self-adaptive mediator identification and formal de-confounding techniques.

Summary updated

Abstract

Although the Conditional Variational Auto-Encoder (CVAE) model can generate more diversified responses than the traditional Seq2Seq model, the responses often have low relevance with the input words or are illogical with the question. A causal analysis is carried out to study the reasons behind, and a methodology of searching for the mediators and mitigating the confounding bias in dialogues is provided. Specifically, we propose to predict the mediators to preserve relevant information and auto-regressively incorporate the mediators into generating process. Besides, a dynamic topic graph guided conditional variational auto-encoder (TGG-CVAE) model is utilized to complement the semantic space and reduce the confounding bias in responses. Extensive experiments demonstrate that the proposed model is able to generate both relevant and informative responses, and outperforms the state-of-the-art in terms of automatic metrics and human evaluations.
BibTeX
@inproceedings{liu2022incorporating,
  author = {Jiayi Liu and Wei Wei and Zhixuan Chu and Xing Gao and Ji Zhang and Tan Yan and Yulin Kang},
  title = {{Incorporating Casual Analysis into Diversified and Logical Response Generation}},
  booktitle = {International Conference on Computational Linguistics},
  year = {2022},
}
Figure