Retrieve and Refine: Improved Sequence Generation Models For Dialogue

Jason Weston, Emily Dinan, Alexander H. Miller

Introduction

The "Retrieve and Refine" (RetNRef) framework addresses a fundamental trade-off in automated dialogue systems between the flexibility of generative models and the richness of retrieval-based systems. Standard generative models, such as Sequence-to-Sequence (Seq2Seq) architectures, often suffer from the "generic response" problem, producing safe but dull outputs like "I don't know." Conversely, retrieval models provide high-quality human responses but are limited by the fixed nature of their database, leading to potential irrelevance or factual inconsistency in specific contexts. RetNRef bridges this gap by using a retrieved response as a "rough draft" for a generator to refine, combining human-like variety with situational flexibility.

Methodology

The RetNRef model employs a two-stage pipeline. First, a Retriever—specifically a Key-Value Memory Network (KV-MemNet)—identifies the most relevant response from the training set based on the current dialogue history. Second, a Generator (a 2-layer LSTM with attention) receives the dialogue history concatenated with the retrieved response, separated by a special token.

The authors developed two advanced variants:

  • RetNRef+: This version "clips" the dialogue history to force the generator to rely more heavily on the retrieved context, preventing the model from defaulting to generic generative patterns.
  • RetNRef++: To address instances where the generator made minor but detrimental edits to high-quality retrievals, this variant implements a heuristic: if the generated output shares more than 60% word overlap with the retrieval, the model defaults to copying the retrieval exactly.

During training, a "look-ahead" strategy was used where retrieval results were reranked based on similarity to the ground-truth label, ensuring the generator learned to effectively refine "near-miss" candidates.

Experimental Setup

Models were evaluated using the ConvAI2 dataset, derived from PersonaChat. This dataset involves "chit-chat" between speakers with assigned personas, comprising approximately 160,000 utterances across 11,000 dialogues. Evaluation focused on engagingness, fluency, and the ability to maintain persona consistency.

Results and Analysis

Lexical Diversity and Vocabulary

A key advantage of the hybrid approach is its ability to act as a "vocabulary booster." While standard Seq2Seq models tend to use a restricted set of common words, the RetNRef models leverage the diverse vocabulary of retrieved human responses. As shown in the lexical analysis, RetNRef++ utilizes rare words at a rate significantly closer to human levels than vanilla Seq2Seq models.

Lexical statistics showing word counts and rare word usage across models.

Human Evaluation and Engagingness

Human-centric metrics reveal that the RetNRef++ model significantly outperforms standalone baselines. In terms of engagingness, RetNRef++ achieved a score of 3.80 on a 5-point scale, surpassing both the vanilla Seq2Seq (2.70) and the pure Memory Network retriever (3.66).

Human evaluation scores for engagingness, fluency, consistency, and persona.

Furthermore, head-to-head A/B testing showed that humans preferred RetNRef++ over the standard Memory Network in 54.5% of cases. Interestingly, the authors noted that traditional metrics like perplexity (PPL) were poor predictors of human preference; the best-performing models often exhibited higher perplexity while being judged as far more engaging by human evaluators.

Conclusion and Future Work

The Retrieve and Refine approach successfully mitigates the dullness of generative models by grounding them in retrieved human responses. However, challenges remain, including occasional phrase repetition and lapses in persona consistency. Future research may explore more integrated attention mechanisms to better balance dialogue history with retrieved information, moving beyond simple concatenation.

Summary updated

BibTeX
@inproceedings{weston2018retrieve,
  author = {Jason Weston and Emily Dinan and Alexander H. Miller},
  title = {{Retrieve and Refine: Improved Sequence Generation Models For Dialogue}},
  booktitle = {SCAI @ EMNLP},
  year = {2018},
  doi = {10.18653/v1/w18-5713},
}
Figure