Abstract
Existing knowledge-grounded open-domain dialogue generation models often face the hallucination problem, where the model persists in using inappropriate knowledge and generates responses inconsistent with facts. This issue stems from polarized optimization objectives and weak knowledge selection abilities. To mitigate this, we propose the Augmentative and Contrastive Knowledge Dialogue Expansion Framework (ACK-DEF). ACK-DEF constructs dialogue samples with varying degrees of knowledge errors and manually designed responses to smooth the optimization objective. Experimental results on the Wikipedia of Wizard dataset demonstrate that ACK-DEF effectively alleviates hallucinations.
Introduction
Knowledge-Grounded Dialogue (KGD) models aim to provide informative responses by leveraging external knowledge sources like Wikipedia. However, these models frequently "hallucinate"—generating factually incorrect or inconsistent content with high confidence. This behavior often arises because models are typically trained on "gold" knowledge, making them overly reliant on provided snippets even when they are noisy or irrelevant.
The authors identify "polarized optimization" as a root cause: models are usually trained on either perfect knowledge or no knowledge at all. In practice, retrieval systems often return imperfect data. Without training on "almost right" or "completely wrong" information, models fail to develop the skepticism needed to handle real-world noise.
Methodology: The ACK-DEF Framework
The Augmentative and Contrastive Knowledge Dialogue Expansion Framework (ACK-DEF) introduces a spectrum of knowledge quality to "smooth" the training process through two primary expansion techniques.
1. Augmentative Knowledge (AK)
AK focuses on improving model generalization by introducing lexical variety. By replacing words in the original knowledge with synonyms, the framework creates two types of samples:
- ak-less: 10-20% word replacement, teaching the model to recognize facts across different phrasings.
- ak-more: 30-50% word replacement. To handle this higher noise level, the framework concatenates the response with manual prefix and post-prompts (e.g., "I was thinking that perhaps...") to encourage more tactful, less certain responses.
2. Contrastive Knowledge (CK)
CK focuses on boundary setting by introducing factual errors. Words are replaced with antonyms to create "incorrect" facts. Instead of forcing the model to use this misinformation, these samples are paired with euphemistic "safety" responses (e.g., "I have never heard of that. What is that one about?"). This teaches the model to ignore suspicious knowledge and fall back on safe communication patterns.
Evaluation and Results
The framework was evaluated using the Wikipedia of Wizard (WoW) dataset, testing on both seen and unseen topics across four test sets: normal, ak-less, ak-more, and ck. Using PLATO-v1 as a base model, the researchers found that ACK-DEF significantly improved robustness:
- Robustness to Noise: On the
ak-moreandcktest sets, ACK-DEF maintained higher coherence and BLEU scores than the baseline, which frequently hallucinated when faced with noise. - Knowledge Filtering: In the
ckset, the model's knowledge recall dropped significantly. This is a positive result, indicating the model successfully ignored incorrect facts rather than incorporating them into the response. - Hallucination Reduction: Human evaluations showed a dramatic decrease in hallucinations. For the
cktest set, the baseline hallucinated 21.67% of the time, whereas ACK-DEF reduced this to 5.67%.
Discussion and Conclusion
ACK-DEF demonstrates that teaching a model to "doubt" its input is crucial for reducing hallucinations. By moving away from binary training objectives to a gradient of knowledge quality, models become more robust to the errors inherent in retrieval systems. This human-inspired approach—using euphemisms when information is unclear—provides a practical path toward more reliable and conversational AI.
Future work includes scaling the framework to larger datasets, testing on diverse backbone architectures, and automating the generation of safety responses using Large Language Models (LLMs).
Summary updated