Training Neural Response Selection for Task-Oriented Dialogue Systems

Matthew Henderson, Ivan Vulic, Daniela Gerz, Iñigo Casanueva, Pawel Budzianowski, Sam Coope, Georgios Spithourakis, Tsung-Hsien Wen, Nikola Mrksic, Pei-Hao Su

Abstract

Despite their popularity in the chatbot literature, retrieval-based models have had modest impact on task-oriented dialogue systems, with the main obstacle to their application being the low-data regime of most task-oriented dialogue tasks. Inspired by the recent success of pretraining in language modelling, we propose an effective method for deploying response selection in task-oriented dialogue. To train response selection models for task-oriented dialogue tasks, we propose a novel method which: 1) pretrains the response selection model on large general-domain conversational corpora; and then 2) fine-tunes the pretrained model for the target dialogue domain, relying only on the small in-domain dataset to capture the nuances of the given dialogue domain. Our evaluation on six diverse application domains, ranging from e-commerce to banking, demonstrates the effectiveness of the proposed training method.

Introduction

Building effective task-oriented dialogue systems, such as customer support bots, is challenging due to the "low-data regime." Unlike general-purpose chatbots that can leverage vast internet data, specialized bots (e.g., for banking or technical support) must understand niche terminology with very few training examples.

Historically, two main approaches have been used:

  1. Generative Models: These can produce novel responses but are prone to "hallucinations" or defaulting to generic, unhelpful phrases.
  2. Retrieval Models: These select the best response from a fixed set. While more controllable and informative, they typically require large amounts of in-domain data to learn domain-specific nuances.

This research bridges the gap by leveraging large-scale general-domain conversations to provide a linguistic foundation before fine-tuning on specific tasks.

Methodology

The proposed framework consists of a two-step training procedure: large-scale pretraining followed by domain-specific fine-tuning.

1. Response Selection Pretraining

The model is first trained on a massive corpus of over 727 million comment-response pairs from Reddit. It employs a dual-encoder architecture to map both the input query and candidate responses into a shared 512-dimensional semantic space.

The dual-encoder architecture used for response selection, featuring self-attention and n-gram embeddings.

The architecture combines unigram and bigram embeddings with positional encoding and self-attention. A key finding was the effectiveness of the swish activation function ($x \cdot \text{sigmoid}(x)$) over standard ReLU or tanh. Scoring is performed using scaled cosine similarity, which facilitates better thresholding for response selection.

2. Target Domain Fine-Tuning

After pretraining, the model is adapted to specific domains using two strategies:

  • FT-DIRECT: Continued training of the Reddit-pretrained model on the target dataset.
  • FT-MIXED: Training on a mixture of target data and Reddit data (typically a 3:1 ratio) to maintain general language capabilities while learning domain specifics.

Experimental Setup

The framework was evaluated across six diverse domains:

  • REDDIT: The source dataset for pretraining.
  • OPENSUBTITLES: General movie/TV dialogue.
  • AMAZONQA: E-commerce Q&A.
  • UBUNTU: Technical support.
  • BANKING: A new FAQ-style dataset for e-banking with 77 intent categories.
  • SEMEVAL15: Lifestyle and tourism information.

Results and Analysis

The two-step approach significantly outperformed baselines, including BM25 and state-of-the-art encoders like BERT, ELMo, and USE.

Quantitative Performance

The gains from fine-tuning were most pronounced in specialized domains. In the Banking domain, fine-tuning resulted in a +67.5% improvement in Recall@1 over the Reddit-only model. The Ubuntu domain saw a +32.5% improvement. Furthermore, the dual-encoder design enabled a 130x speedup in retrieval through pre-computed embeddings and approximate nearest neighbor search.

Qualitative Insights

The model's ability to distinguish between subtle intents is best visualized through t-SNE embeddings. After fine-tuning, the model forms much tighter and more distinct clusters for specific categories compared to general-purpose encoders.

t-SNE visualizations comparing different models; fine-tuning (FT-MIXED) creates significantly more coherent intent clusters.

Key Insights

  • Mapping vs. Similarity: Unlike standard encoders that seek semantic similarity, this model learns the mapping from a question to a valid response.
  • N-gram Importance: Ablation studies showed that combining n-grams with self-attention provides a substantial performance boost over simpler models.
  • Data Efficiency: The model achieves significant gains even with small target datasets, demonstrating that Reddit pretraining provides a robust linguistic foundation.

Conclusion and Future Work

This research demonstrates that pretraining on large-scale conversational data followed by domain-specific fine-tuning is a highly effective strategy for task-oriented dialogue. Future work may explore "hard" negative sampling to further improve accuracy and the incorporation of deeper multi-turn context. Additionally, balancing the trade-off between task specialization and general language retention remains an area for optimization.

Summary updated

Abstract

Despite their popularity in the chatbot literature, retrieval-based models have had modest impact on task-oriented dialogue systems, with the main obstacle to their application being the low-data regime of most task-oriented dialogue tasks. Inspired by the recent success of pretraining in language modelling, we propose an effective method for deploying response selection in task-oriented dialogue. To train response selection models for task-oriented dialogue tasks, we propose a novel method which: 1) pretrains the response selection model on large general-domain conversational corpora; and then 2) fine-tunes the pretrained model for the target dialogue domain, relying only on the small in-domain dataset to capture the nuances of the given dialogue domain. Our evaluation on six diverse application domains, ranging from e-commerce to banking, demonstrates the effectiveness of the proposed training method.
BibTeX
@inproceedings{henderson2019training,
  author = {Matthew Henderson and Ivan Vulic and Daniela Gerz and Iñigo Casanueva and Pawel Budzianowski and Sam Coope and Georgios Spithourakis and Tsung-Hsien Wen and Nikola Mrksic and Pei-Hao Su},
  title = {{Training Neural Response Selection for Task-Oriented Dialogue Systems}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2019},
  url = {https://openreview.net/forum?id=RO8NvOhhinJ},
}
Figure