TL;DR
This paper investigates "hallucinations" – generating entities not present in the source image – in visual dialogue agents playing the GuessWhat?! game. They adapt advanced multimodal Transformer models (LXMERT-GDSE and VLP) and find they significantly reduce hallucinations compared to older models, with VLP performing closest to human levels. This highlights the critical role of robust visual processing in preventing these errors.
What Problem Does This Paper Address?
Natural language generation (NLG) systems, while impressive, often struggle with "hallucinations," where they generate text that is inconsistent with the input. This is particularly problematic in multimodal systems that combine vision and language, as it can lead to nonsensical or misleading outputs. The GuessWhat?! game, a referential visual dialogue task where a questioner tries to identify a target object in an image by asking yes/no questions to an oracle, serves as a testbed for this issue. While previous work has explored hallucinations in tasks like summarization and image captioning, their impact and mitigation in the complex, interactive setting of visual dialogue agents remain understudied. This paper aims to quantify and understand entity hallucinations in GuessWhat?! agents and explore how different visual processing approaches affect their occurrence.
Methods
The authors adapt and compare four Questioner models for the GuessWhat?! task:
- BL (Baseline): A recurrent neural network (RNN) with an LSTM transition function.
- GDSE (Visually-Grounded Dialogue State Encoder): A model that uses visual features and dialogue history to create a shared representation for question generation.
- LXMERT-GDSE: A new model based on the LXMERT multimodal Transformer, adapted for the GuessWhat?! Questioner role.
- VLP (Unified Vision-Language Pre-training): A powerful multimodal Encoder-Decoder Transformer architecture pre-trained on image captioning, fine-tuned for GuessWhat?!.
They use the CHAIR metric (Caption Hallucination Assessment with Image Relevance) to quantify hallucinations. This metric has two variants: CHAIR-i (per-instance) and CHAIR-s (per-sentence). They also analyze the dialogue accuracy and perform a qualitative analysis of hallucination occurrences.
Datasets
The primary dataset used is GuessWhat?!. This dataset consists of over 150,000 human-human dialogues generated on MSCOCO images, with an average of 5.3 questions per dialogue. The authors use the test set of this dataset for evaluation.
Key Insights and Ideas
- Visual Processing is Key: The most significant finding is that models with more sophisticated and fine-grained visual processing (LXMERT-GDSE and VLP) drastically reduce hallucinations compared to older models like BL and GDSE. This confirms that a strong understanding of the visual input is crucial for grounding generated text.
- Hallucinations Cascade: Hallucinated entities tend to appear in consecutive turns, suggesting a "cascade effect" where one hallucination can trigger others. This is observed in machine-generated dialogues but not as strongly in human ones.
- Negation is Tricky: Hallucinations occur much more frequently after negative answers ("No") than positive ones ("Yes"). This points to a potential difficulty for multimodal models in grounding negation, which is a known challenge in NLP.
- Accuracy Isn't Everything: The paper highlights that simple task accuracy (guessing the object correctly) is a poor proxy for dialogue quality. The baseline model, despite generating many hallucinations, sometimes achieves higher accuracy due to a reliance on location-based questions, which can make dialogues unnatural.
- Human-like vs. Real Hallucinations: Not all hallucinations are equal. Some are "fake" (e.g., referring to a common object like a table that's implied but not explicitly labeled in the dataset), while others are "real" (entities truly not present). VLP generates more human-like, often fake, hallucinations, while other models produce more "real" ones.
Key Results
- CHAIR Scores: LXMERT-GDSE and VLP generated less than half the hallucinations of BL and GDSE. VLP achieved the lowest CHAIR-i score (7.45) and CHAIR-s score (10.78), coming closest to human performance (CHAIR-i: 4.11, CHAIR-s: 7.45).
- Accuracy Discrepancy: While VLP and LXMERT-GDSE had fewer hallucinations, the baseline model (BL) sometimes showed higher accuracy. This was attributed to BL's strategy of asking many location-based questions, which are effective for guessing but can lead to unnatural dialogues.
- Per-Turn Analysis: Hallucinations were less frequent in earlier dialogue turns and tended to increase towards the end.
- Consecutive Hallucinations: VLP and human dialogues had the lowest percentage of consecutive hallucinations (8.09% and 8.09% respectively), indicating a weaker cascade effect. BL had the highest (24.13%).
- Qualitative Analysis: VLP's hallucinations were often "fake" (due to annotation limitations or common sense inferences), making them more human-like. BL and GDSE generated more "real" hallucinations.
Limitations and Future Work
- Annotation Limitations: The CHAIR metric relies on MSCOCO segmentation, which is not exhaustive. This leads to "fake" hallucinations, especially when humans refer to objects implied by common sense or context.
- Attribute Hallucination: The paper focuses on entity hallucinations and leaves attribute hallucination (e.g., "is it green?" when the object is not green) for future work.
- Negation Understanding: The difficulty in grounding negative answers remains a significant challenge. Further research is needed to understand the role of negation in visual dialogues.
- Decoding Strategies: Developing new decoding strategies for NLG systems is suggested to further reduce hallucinations.
- Hallucination vs. Repetition: The interplay between hallucination and repetition, another issue affecting machine-generated dialogue quality, warrants further investigation.
Summary updated