TL;DR
This paper tackles the challenge of making knowledge-grounded dialogue systems more truthful and less prone to "hallucinating" information. They introduce novel evaluation metrics and control mechanisms for generative models, demonstrating that these techniques significantly improve the faithfulness and objectivity of dialogue responses, as confirmed by human evaluations.
What Problem Does This Paper Address?
Building dialogue systems that can accurately convey information from external documents (knowledge-grounded dialogue) is a significant challenge. While current generative models can mimic human language, they often struggle to stay faithful to the provided evidence. Existing datasets, like the Wizard of Wikipedia, contain a mix of factual and subjective responses, making it difficult to train models that only produce verifiable information. This can lead to systems that "hallucinate" – generating information not present in the source document, or even fabricating details. The authors aim to bridge this gap by developing methods to control generative models, ensuring their responses are both informative and truthful.
Methods
The core of the paper lies in developing and applying controllable generation techniques. They propose three key evaluation measures to assess response quality:
- Objective Voice: Detecting the presence of first-person pronouns to identify subjective statements.
- Lexical Precision: Measuring how many words in the response are present in the evidence document.
- Entailment: Using a pre-trained Natural Language Inference (NLI) model to check if the response is logically supported by the evidence.
These measures are then integrated into the training and generation process in two ways:
- Control Code Features: Special tokens representing the desired levels of objectivity, precision, and entailment are prepended to the input sequence during training. At generation time, these tokens guide the model towards producing responses that meet these criteria.
- Controlled Resampling: At decoding time, the system repeatedly samples responses until one is found that satisfies the desired evaluation measures. This acts as a post-hoc filter.
The authors fine-tune pre-trained seq2seq models (GPT-2 and T5) on the Wizard of Wikipedia dataset using these methods.
Datasets
The primary dataset used is Wizard of Wikipedia (Dinan et al., 2019). This dataset consists of multi-turn dialogues where one participant (the "wizard") has access to Wikipedia documents and is expected to provide information based on them. The authors use the training portion of this dataset for model training and the development/test sets for evaluation. They note that while this dataset is knowledge-grounded, it contains a significant number of subjective and non-factual utterances, which is precisely the problem they aim to address.
Key Insights and Ideas
- Disentangling Response Styles is Crucial: The presence of both factual and subjective responses in existing datasets is a major hurdle for training truthful dialogue agents. Simply training on such data leads to models that inherit these mixed styles.
- Evaluation Metrics as Control Levers: The proposed metrics (objective voice, lexical precision, entailment) are not just for evaluation but can be directly used as control signals to steer generative models.
- Control Codes vs. Resampling: Control codes offer a more integrated approach during training, while resampling provides a direct, albeit potentially less efficient, way to enforce constraints at generation time. Combining both can yield the best results.
- Trade-offs Exist: While improving faithfulness and objectivity, there can be a slight decrease in response relevance, highlighting a common trade-off in controlled generation.
Key Results
- Significant Improvement in Faithfulness and Objectivity: Both control code features and resampling methods, especially when used together, significantly improve the faithfulness and objectivity of generated responses compared to baseline models. Human evaluations confirmed this, showing higher scores for these aspects.
- Human Judgments Correlate with Metrics: The proposed automatic metrics (objective voice, lexical precision, entailment) showed strong correlations with human judgments of faithfulness and objectivity, validating their effectiveness.
- T5 Models Show Strong Performance: The T5 models, particularly when augmented with control codes and resampling, consistently outperformed baselines across various metrics.
- Resampling Alone is Less Effective: While resampling can improve metrics, it's less effective on its own compared to control codes, likely due to the difficulty of finding satisfactory responses within a limited number of attempts.
Limitations and Future Work
- Relevance vs. Faithfulness Trade-off: The authors acknowledge that improving faithfulness and objectivity can sometimes come at the cost of response relevance.
- Extractive Nature: In some cases, the controlled models can become too extractive, including irrelevant details from the evidence.
- Inference Challenges: The models still struggle with questions that require significant inference rather than direct extraction from the evidence.
- Future Work: The authors suggest exploring approaches that combine extractive and abstractive generation methods more deliberately to better integrate evidence with conversational context. They also point to the need for more semantic-oriented measures of precision.
Summary updated