PolyResponse: A Rank-based Approach to Task-Oriented Dialogue with Application in Restaurant Search and Booking

Matthew Henderson, Ivan Vulic, Iñigo Casanueva, Pawel Budzianowski, Daniela Gerz, Sam Coope, Georgios Spithourakis, Tsung-Hsien Wen, Nikola Mrksic, Pei-Hao Su

Abstract

Matthew Henderson, Ivan Vulić, Iñigo Casanueva, Paweł Budzianowski, Daniela Gerz, Sam Coope, Georgios Spithourakis, Tsung-Hsien Wen, Nikola Mrkšić, Pei-Hao Su. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP): System Demonstrations.

2019.

Introduction

PolyResponse is a retrieval-based dialogue system designed to overcome the limitations of traditional task-oriented architectures. By replacing rigid, manually-engineered ontologies with a ranking model trained on hundreds of millions of real-world conversations, PolyResponse treats dialogue as a large-scale similarity search problem. This approach enables multi-modal (text and image) restaurant search and booking across eight languages without requiring traditional natural language generation (NLG).

The Challenge of Traditional Dialogue Systems

Traditional task-oriented dialogue systems, such as those used for booking flights or restaurant tables, typically follow a modular pipeline: Natural Language Understanding (NLU), State Tracking, and NLG. This architecture relies on a rigid ontology—a predefined list of "slots" (e.g., price_range, cuisine) and "values" (e.g., cheap, Italian).

This approach suffers from several drawbacks:

  1. High Cost: Engineering ontologies and annotating domain-specific data is labor-intensive and slow.
  2. Brittleness: Systems often fail when user queries fall outside the narrow bounds of the predefined ontology.
  3. Data Scarcity: While general conversational data is abundant, high-quality annotated task-oriented data is scarce and small in scale.

PolyResponse bypasses these hurdles by using a retrieval-based approach that learns implicit semantics from massive, unlabelled conversational datasets.

Methodology: Dual Encoder Ranking

The core of PolyResponse is a Dual Encoder ranking model. Instead of generating text word-by-word, the system encodes the conversation history (context) and a pool of potential responses into a shared vector space.

The PolyResponse architecture uses dual encoders to map contexts, replies, and photos into a shared vector space, trained on Reddit and Yelp data.
  • Text Representation: A transformer-based architecture maps unigrams and bigrams into 512-dimensional vectors. The model is trained to maximize the cosine similarity between a context ($h_c$) and its correct reply ($h_r$) while minimizing similarity with random "negative" examples.
  • Multi-modal Integration: Images are treated as responses. Using a pre-trained MobileNet CNN, photos are encoded into the same 512-dimensional space ($h_p$), allowing the system to rank images based on their relevance to the conversation context.
  • Efficient Retrieval: To handle millions of potential responses in real-time, the system utilizes Faiss, a library for efficient similarity search, enabling scaling to billion-sized indices.

Datasets

The model leverages large-scale, diverse datasets to learn natural human interaction:

  • Reddit Corpus: The primary training set, consisting of 3.7 billion comments (727 million context-reply pairs). This provides a broad understanding of natural human interaction.
  • Yelp Open Dataset: Used for the multi-modal component, containing 200,000 photos and captions, allowing the model to learn the relationship between visual content and descriptive text.
  • Domain-Specific Indices: For the demonstration, the system indexes ~125,000 sentences from online reviews and ~6,700 responses derived from structured restaurant data.

Multilingual and Multi-modal Capabilities

By training on massive datasets like Reddit, the model learns implicit semantics. For example, it recognizes that "Is it pricey?" is semantically related to "It's quite affordable" without an explicit price slot. Treating images as rankable responses provides a computationally elegant way to achieve multi-modality, allowing the system to "show" rather than just "tell."

PolyResponse demonstrates zero-shot multilinguality across various languages and cities, including English, French, German, and Chinese.

The system supports eight languages (English, German, Spanish, Mandarin, Polish, Russian, Korean, and Serbian) using a "translate-to-source" approach. User input is translated to English on-the-fly, processed by the English-trained model, and responses are retrieved from a pre-translated index.

Conclusion and Future Work

PolyResponse demonstrates a scalable, functional approach to dialogue that handles voice-controlled menu search and multilingual queries. However, limitations remain. The current multilingual support relies on external translation services at inference time, and as a retrieval-based system, it cannot generate novel responses outside its index. Future work will explore native multilingual encoders and more complex, learned dialogue flows to replace current heuristic policies.

Summary updated

BibTeX
@inproceedings{henderson2019polyresponse,
  author = {Matthew Henderson and Ivan Vulic and Iñigo Casanueva and Pawel Budzianowski and Daniela Gerz and Sam Coope and Georgios Spithourakis and Tsung-Hsien Wen and Nikola Mrksic and Pei-Hao Su},
  title = {{PolyResponse: A Rank-based Approach to Task-Oriented Dialogue with Application in Restaurant Search and Booking}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2019},
  url = {https://openreview.net/forum?id=s5GNuYJdMZG},
}
Figure