Abstract
Ze Yang, Wei Wu, Jian Yang, Can Xu, Zhoujun Li. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP).
2019.
Introduction
Modern open-domain chatbots typically rely on encoder-decoder architectures trained on massive datasets of message-response pairs. However, a significant challenge arises when developers wish to constrain responses to specific styles, emotions, or intents—a scenario often referred to as "data hunger." While billions of social media posts exist, finding millions of pairs that adhere to specific constraints (e.g., "always respond with a question") is difficult. For instance, in a public dataset of 5 million Weibo sessions, only 7.3% contain question-response pairs. Standard models often overfit or produce generic, irrelevant responses when trained on such sparse data. This paper addresses the challenge of leveraging large volumes of unpaired text to improve response generation in low-resource settings.
Methodology: The S2S-Temp Framework
The authors propose S2S-Temp, a framework that utilizes structural "templates" learned from unpaired data to guide response generation. The approach consists of three primary components:
1. Unsupervised Template Induction
The model employs a Neural Hidden Semi-Markov Model (NHSMM) to distill templates from a large corpus of unpaired data ($D_U$). Unlike a standard Hidden Markov Model (HMM), the semi-Markov variant allows hidden states to emit sequences of words of varying lengths. This captures the underlying "skeleton" of a sentence (e.g., [Pronoun] [Verb] [Noun] [?]) rather than just word-to-word transitions.
2. Response Generation with Template Prior
These learned templates are integrated into an encoder-decoder model. During generation, the model samples a template—a sequence of latent states—which serves as a structural prior. The decoder then fills in these states based on the input message context.
3. Adversarial Learning
Training on small paired datasets ($D_P$) using Maximum Likelihood Estimation (MLE) often results in a trade-off: the model may ignore the input message to strictly follow the template or ignore the template to overfit the small paired data. To mitigate this, the authors use a Generative Adversarial Network (GAN) approach. A discriminator distinguishes between human and model-generated responses, providing a reward signal via REINFORCE that encourages the generator to be both structurally sound and contextually relevant.
Experimental Setup
The model was evaluated on two distinct tasks:
- Question Response Generation: Utilizing 481k Weibo pairs and 776k unpaired questions from Zhihu.
- Sentiment Response Generation: Utilizing 360k Twitter pairs (positive sentiment) and 1 million unpaired positive tweets.
Results and Discussion
S2S-Temp significantly outperformed several baselines, including standard Seq2Seq, Conditional Variational Autoencoders (CVAE), and task-specific models like the Emotional Chatting Machine (ECM).
Automatic Evaluation
In the question-generation task, S2S-Temp achieved a BLEU-1 score of 0.102, a substantial improvement over Seq2Seq's 0.037. It also demonstrated superior performance in semantic similarity metrics such as "Embedding Greedy" and "Extrema."
Human Evaluation
Side-by-side human comparisons confirmed that S2S-Temp produces responses with higher fluency, relevance, and richness. As shown in the human evaluation results, S2S-Temp was consistently preferred over baselines. For example, in sentiment generation, S2S-Temp was preferred 48.4% of the time compared to only 9.0% for CVAE.
Conclusion
The S2S-Temp framework demonstrates that the semantic and syntactic knowledge embedded in unpaired data is a powerful substitute for missing conversation pairs. By using explicit latent states, the model also offers improved interpretability, as specific states map to functional roles like pronouns or punctuation. While the architecture is more complex than standard Transformers—requiring two-stage training and adversarial loops—it provides a robust solution for stylized response generation in data-sparse environments. Future work includes extending the model to multi-turn conversations and refining the template sampling process.
Summary updated