Low-Resource Response Generation with Template Prior

Ze Yang, Wei Wu, Jian Yang, Can Xu, Zhoujun Li

Abstract

We study open domain response generation with limited message-response pairs. The problem exists in real-world applications but is less explored by the existing work. Since the paired data now is no longer enough to train a neural generation model, we consider leveraging the large scale of unpaired data that are much easier to obtain, and propose response generation with both paired and unpaired data. The generation model is defined by an encoder-decoder architecture with templates as prior, where the templates are estimated from the unpaired data as a neural hidden semi-markov model. By this means, response generation learned from the small paired data can be aided by the semantic and syntactic knowledge in the large unpaired data. To balance the effect of the prior and the input message to response generation, we propose learning the whole generation model with an adversarial approach. Empirical studies on question response generation and sentiment response generation indicate that when only a few pairs are available, our model can significantly outperform several state-of-the-art response generation models in terms of both automatic and human evaluation.

Introduction

Modern open-domain dialogue systems typically rely on massive datasets of message-response pairs to train encoder-decoder architectures. However, in many real-world scenarios, such paired data is scarce, particularly for specific conversational styles or niche domains. This "data sparsity" problem often leads to models that produce generic, repetitive, or ungrammatical responses. While paired data may be limited, large-scale unpaired text—such as social media posts or forum questions—is often abundant. This paper introduces S2S-Temp, a model designed to leverage this unpaired data as a structural "template prior" to improve response generation in low-resource settings.

Methodology

The S2S-Temp framework consists of two primary stages: unsupervised template distillation and template-guided response generation.

Unsupervised Template Distillation

The authors utilize a Neural Hidden Semi-Markov Model (NHSMM) to learn latent structures from a large corpus of unpaired data. Unlike standard Hidden Markov Models, the NHSMM allows each hidden state to emit a sequence of words of variable length. This enables the model to automatically capture the underlying "skeletons" or templates of typical responses (e.g., the structure of a question or a positive sentiment statement) without requiring manual labeling.

S2S-Temp Architecture

The generation model is an encoder-decoder architecture that incorporates the learned templates as a prior. As shown in Figure 1, the encoder processes the input message, while the decoder is guided by a sequence of latent states ($z$) and durations ($l$) sampled from the template prior.

Architecture of the S2S-Temp model showing the encoder-decoder structure with template-guided states.

The decoder consists of two layers: a state-level GRU ($GRU_H$) that manages the transition between template states, and a word-level GRU ($GRU_Y$) that generates the actual tokens. This hierarchical structure ensures that the generated response adheres to the structural constraints of the sampled template while remaining flexible enough to incorporate message-specific content.

Adversarial Learning

To balance the influence of the template prior with the context of the input message, the authors employ a Generative Adversarial Network (GAN) framework. The S2S-Temp model acts as the generator, while a CNN-based discriminator is trained to distinguish between human-authored responses and those generated by the model. This adversarial objective encourages the generator to produce responses that are both structurally sound and contextually relevant, avoiding the pitfalls of standard Maximum Likelihood Estimation (MLE) on small datasets.

Experiments and Results

The model was evaluated on two low-resource tasks: Question Response Generation (using Weibo and Zhihu data) and Sentiment Response Generation (using Twitter data).

Automatic Evaluation

S2S-Temp was compared against several baselines, including standard Seq2Seq, Conditional Variational Autoencoders (CVAE), and task-specific models like the Emotional Chatting Machine (ECM) and Hidden Topic-based Decoder (HTD).

Automatic evaluation results for the Question Response Generation task.

As shown in Figure 3, S2S-Temp significantly outperformed the baselines across all metrics in the question-generation task. Notably, it achieved a BLEU-1 score of 0.102, nearly triple that of the standard Seq2Seq model (0.037). Ablation studies (e.g., S2S-Temp-None, which lacks the template prior) further demonstrated that the structural knowledge transferred from unpaired data is critical for high performance.

Human Evaluation

Human judges assessed the generated responses based on Fluency, Relevance, and Richness. S2S-Temp consistently received higher ratings than the baselines. For instance, in the sentiment task, S2S-Temp was judged more relevant than the ECM baseline in 36.9% of cases, while ECM was preferred in only 13.9%.

Discussion and Conclusion

The success of S2S-Temp highlights the value of using templates as "structural anchors" in neural generation. By distilling grammar and syntax from raw, unpaired text, the model overcomes the limitations of small paired datasets. The adversarial training further ensures that these templates do not "overwhelm" the input message, maintaining a balance between structure and relevance.

Future work may explore extending this framework to multi-turn conversations and developing more sophisticated methods for selecting the most appropriate template based on the input message's semantics.

Summary updated

Abstract

We study open domain response generation with limited message-response pairs. The problem exists in real-world applications but is less explored by the existing work. Since the paired data now is no longer enough to train a neural generation model, we consider leveraging the large scale of unpaired data that are much easier to obtain, and propose response generation with both paired and unpaired data. The generation model is defined by an encoder-decoder architecture with templates as prior, where the templates are estimated from the unpaired data as a neural hidden semi-markov model. By this means, response generation learned from the small paired data can be aided by the semantic and syntactic knowledge in the large unpaired data. To balance the effect of the prior and the input message to response generation, we propose learning the whole generation model with an adversarial approach. Empirical studies on question response generation and sentiment response generation indicate that when only a few pairs are available, our model can significantly outperform several state-of-the-art response generation models in terms of both automatic and human evaluation.
BibTeX
@inproceedings{yang2019low,
  author = {Ze Yang and Wei Wu and Jian Yang and Can Xu and Zhoujun Li},
  title = {{Low-Resource Response Generation with Template Prior}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2019},
  url = {https://openreview.net/forum?id=DLIl4JMpkRw},
}
Figure