HallE-Switch: Rethinking and Controlling Object Existence Hallucinations in Large Vision Language Models for Detailed Caption

Bohan Zhai, Shijia Yang, Xiangchen Zhao, Chenfeng Xu, Sheng Shen, Dongdi Zhao, Kurt Keutzer, Manling Li, Tan Yan, Xiangjun Fan

Abstract

Large Multimodal Models (LMMs) frequently hallucinate objects in detailed captions by over-relying on parametric world knowledge when visual inputs are ambiguous. This paper introduces CCEval, a GPT-4-based evaluation framework for captioning, and HallE-Control, a novel architecture featuring a "hallucination knob." This control mechanism allows users to toggle between strict visual grounding and creative inference, effectively managing the tension between contextual and parametric knowledge.

Introduction

While current Large Multimodal Models (LMMs), such as LLaVA, excel at simple visual question answering (VQA), they often struggle with "detailed captioning"—the task of describing a scene in depth. In these scenarios, models frequently suffer from Object Existence Hallucination, claiming objects are present when they are not visually verifiable.

The authors argue that existing benchmarks like POPE are insufficient because they rely on binary "yes/no" questions. They identify a fundamental tension: a model's contextual knowledge (visual pixels) often conflicts with its parametric knowledge (learned world associations). For instance, seeing a dining table might trigger a "guess" that chairs are present, even if they are occluded or beyond the vision encoder's resolution.

Methodology

CCEval: Caption Concept Matching Evaluation

To move beyond simple VQA, the authors developed CCEval. This framework utilizes GPT-4 to extract individual objects from model-generated captions and compare them against ground-truth labels. A key metric introduced is Coverage, which ensures models do not simply produce short, vague captions to artificially lower hallucination rates.

HallE-Control: The Hallucination Knob

Rather than attempting to eliminate hallucinations entirely—which can sometimes be "beneficial imagination" (e.g., inferring occluded objects in robotics)—the authors propose HallE-Control. This architecture introduces a lightweight linear "switch" layer before the language model head.

The HallE-Control mechanism allows users to adjust the model's behavior between strict grounding and creative inference using a controllable parameter.

Users can adjust a continuous parameter $\epsilon$ (the "knob") between -1 and 1:

  • $\epsilon = -1$ (No Imagination): The model adheres strictly to objects grounded by the vision module.
  • $\epsilon = 1$ (Max Imagination): The model leverages parametric knowledge to infer likely objects.
  • Marking: In "imagination" mode, inferred objects can be automatically wrapped in brackets (e.g., [person]) to distinguish them from visually grounded ones.

Key Insights and Analysis

  • Resolution as the Bottleneck: Hallucination is primarily driven by the vision encoder's resolution. If the vision module cannot resolve an object, the language model "fills in the blanks" using semantic associations.
  • Data Scaling Paradox: Increasing instruction data (e.g., using the 2.4M SVIT samples) actually increased hallucination rates, suggesting that low-quality training data encourages guessing over seeing.

Key Results

HallE-Control significantly outperforms baselines in balancing detail and accuracy. In the LLaVA-13B implementation, setting $\epsilon = -1$ reduced the hallucination index ($CHAIR_i$) from 16.0 to 6.37, a substantial improvement in reliability.

Quantitative results show that HallE-Control (Switch and Indication) achieves a better trade-off between low hallucination (CHAIRi) and high object coverage compared to baselines.

Compared to the LLaVA-7B baseline, HallE-Control reduced the object hallucination rate by 44% while maintaining high object coverage and sentence length. The authors also demonstrated that models appearing "hallucination-free" on VQA benchmarks (like InstructBLIP) still exhibit significant hallucinations in detailed captioning, validating the need for CCEval.

Conclusion

This work highlights that object existence hallucination is a controllable trade-off between observation and inference. While this study focuses on object existence, future work is needed to address hallucinations regarding object attributes and spatial relationships. The "switch" architecture holds promise for other multimodal tasks where the balance of visual evidence and prior knowledge is critical.

Summary updated

Abstract

Current Large Multimodal Models (LMMs) achieve remarkable progress, yet there remains significant uncertainty regarding their ability to accurately apprehend visual details, that is, in performing detailed captioning. To address this, we introduce $\textit{CCEval}$, a GPT-4 assisted evaluation method for detailed captioning. Interestingly, while LMMs demonstrate minimal object existence hallucination in existing VQA benchmarks, our proposed evaluation reveals continued susceptibility to such hallucinations. In this paper, we make the first attempt to investigate such hallucination from different aspects, including image resolution, the language decoder size, and instruction data amount, quality, granularity. Our findings underscore the unwarranted inference when the language description includes details at a finer object granularity than what the vision module can ground or verify, thus inducing hallucination. To control such hallucinations, we further attribute the reliability of captioning to contextual knowledge (involving only contextually grounded objects) and parametric knowledge (containing inferred objects by the model). Thus, we introduce $\textit{HallE-Control}$, a controllable LMM in terms of $\textbf{Hall}$ucination in object $\textbf{E}$xistence. HallE-Control can condition the captioning to shift between (i) exclusively depicting contextual knowledge for grounded objects and (ii) blending it with parametric knowledge to imagine inferred objects. Our method reduces hallucination by 44% compared to LLaVA$_{7B}$ and maintains the object coverage.
BibTeX
@misc{zhai2023halle,
  author = {Bohan Zhai and Shijia Yang and Xiangchen Zhao and Chenfeng Xu and Sheng Shen and Dongdi Zhao and Kurt Keutzer and Manling Li and Tan Yan and Xiangjun Fan},
  title = {{HallE-Switch: Rethinking and Controlling Object Existence Hallucinations in Large Vision Language Models for Detailed Caption}},
  year = {2023},
  eprint = {2310.01779},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2310.01779},
}
Figure