Abstract
Large Multimodal Models (LMMs) frequently hallucinate objects in detailed captions by over-relying on parametric world knowledge when visual inputs are ambiguous. This paper introduces CCEval, a GPT-4-based evaluation framework for captioning, and HallE-Control, a novel architecture featuring a "hallucination knob." This control mechanism allows users to toggle between strict visual grounding and creative inference, effectively managing the tension between contextual and parametric knowledge.
Introduction
While current Large Multimodal Models (LMMs), such as LLaVA, excel at simple visual question answering (VQA), they often struggle with "detailed captioning"—the task of describing a scene in depth. In these scenarios, models frequently suffer from Object Existence Hallucination, claiming objects are present when they are not visually verifiable.
The authors argue that existing benchmarks like POPE are insufficient because they rely on binary "yes/no" questions. They identify a fundamental tension: a model's contextual knowledge (visual pixels) often conflicts with its parametric knowledge (learned world associations). For instance, seeing a dining table might trigger a "guess" that chairs are present, even if they are occluded or beyond the vision encoder's resolution.
Methodology
CCEval: Caption Concept Matching Evaluation
To move beyond simple VQA, the authors developed CCEval. This framework utilizes GPT-4 to extract individual objects from model-generated captions and compare them against ground-truth labels. A key metric introduced is Coverage, which ensures models do not simply produce short, vague captions to artificially lower hallucination rates.
HallE-Control: The Hallucination Knob
Rather than attempting to eliminate hallucinations entirely—which can sometimes be "beneficial imagination" (e.g., inferring occluded objects in robotics)—the authors propose HallE-Control. This architecture introduces a lightweight linear "switch" layer before the language model head.
Users can adjust a continuous parameter $\epsilon$ (the "knob") between -1 and 1:
- $\epsilon = -1$ (No Imagination): The model adheres strictly to objects grounded by the vision module.
- $\epsilon = 1$ (Max Imagination): The model leverages parametric knowledge to infer likely objects.
- Marking: In "imagination" mode, inferred objects can be automatically wrapped in brackets (e.g.,
[person]) to distinguish them from visually grounded ones.
Key Insights and Analysis
- Resolution as the Bottleneck: Hallucination is primarily driven by the vision encoder's resolution. If the vision module cannot resolve an object, the language model "fills in the blanks" using semantic associations.
- Data Scaling Paradox: Increasing instruction data (e.g., using the 2.4M SVIT samples) actually increased hallucination rates, suggesting that low-quality training data encourages guessing over seeing.
Key Results
HallE-Control significantly outperforms baselines in balancing detail and accuracy. In the LLaVA-13B implementation, setting $\epsilon = -1$ reduced the hallucination index ($CHAIR_i$) from 16.0 to 6.37, a substantial improvement in reliability.
Compared to the LLaVA-7B baseline, HallE-Control reduced the object hallucination rate by 44% while maintaining high object coverage and sentence length. The authors also demonstrated that models appearing "hallucination-free" on VQA benchmarks (like InstructBLIP) still exhibit significant hallucinations in detailed captioning, validating the need for CCEval.
Conclusion
This work highlights that object existence hallucination is a controllable trade-off between observation and inference. While this study focuses on object existence, future work is needed to address hallucinations regarding object attributes and spatial relationships. The "switch" architecture holds promise for other multimodal tasks where the balance of visual evidence and prior knowledge is critical.
Summary updated