Abstract
Current Large Multimodal Models (LMMs) achieve remarkable progress, yet there remains significant uncertainty regarding their ability to accurately apprehend visual details, particularly in detailed captioning. To address this, we introduce CCEval, a GPT-4 assisted evaluation method for detailed captioning. Interestingly, while LMMs demonstrate minimal object existence hallucination in existing VQA benchmarks, our proposed evaluation reveals continued susceptibility to such hallucinations. In this paper, we investigate such hallucination from different aspects, including image resolution, language decoder size, and instruction data quality. Our findings underscore the unwarranted inference when language descriptions include details at a finer granularity than what the vision module can ground, thus inducing hallucination. To control this, we introduce HallE-Control, a controllable LMM that can shift between exclusively depicting contextual knowledge for grounded objects and blending it with parametric knowledge to imagine inferred objects. Our method reduces hallucination by 44% compared to LLaVA-7B while maintaining object coverage.
Introduction: The "Honesty" Problem in LMMs
Large Multimodal Models (LMMs) like LLaVA are increasingly capable of generating rich, descriptive captions for complex scenes. However, they suffer from a persistent "honesty" problem: they frequently hallucinate objects that are not present in the image. This often stems from "unwarranted inference," where the model's internal language weights (parametric knowledge) suggest that certain objects should be present based on context, even if they are occluded or absent. For instance, seeing a dining table might lead a model to describe chairs that are actually out of frame.
Existing benchmarks like POPE often fail to capture this because they rely on simple Yes/No questions. When tasked with detailed captioning, models revert to guessing. HallE-Control addresses this by providing a "knob" to regulate this behavior, allowing users to choose between strict adherence to visual evidence and imaginative reasoning.
Methodology
CCEval: A New Standard for Captioning Accuracy
To accurately measure hallucinations in long-form text, the authors developed CCEval (Caption Concept Matching Evaluation). This GPT-4 assisted pipeline extracts objects from a model's detailed caption and compares them against ground-truth annotations. Unlike traditional metrics like CHAIR, CCEval leverages GPT-4’s reasoning to handle synonyms and contextual nuances, providing a more reliable score for both object coverage and hallucination rates.
The HallE-Control Architecture
HallE-Control is built upon the "LM-Switch" technique. The researchers inserted a lightweight, trainable linear layer ($W$) into the word embedding space of a frozen LMM. The training process involves two distinct types of knowledge:
- Contextual Knowledge: Objects that the vision encoder can explicitly ground in the image.
- Parametric Knowledge: Objects the model "imagines" based on learned word-word associations.
By training on a contrastive dataset, the model learns to respond to a control parameter $\epsilon$. Setting $\epsilon = -1$ forces the model to stick strictly to what it sees (contextual only), while setting $\epsilon = 1$ allows it to blend in its parametric "imagination."
Key Insights and Results
Resolution vs. Model Size
A striking finding of the study is that increasing image resolution (e.g., using 336px crops or sliding windows) is more effective at reducing hallucinations than scaling up the language decoder. If the vision module (the "eyes") cannot resolve a detail, the language model (the "brain") will inevitably start guessing, regardless of its parameter count.
Data Quality Over Quantity
The research suggests that simply increasing the volume of instruction data can be counterproductive. If the training data contains details that the vision module cannot verify, it encourages the model to hallucinate. High-quality, grounded data is essential for maintaining model honesty.
Performance Metrics
In empirical tests, HallE-Control demonstrated a 44% reduction in object existence hallucinations compared to the LLaVA-7B baseline. Furthermore, it achieved a superior Pareto balance between CHAIR (hallucination rate) and Coverage (the breadth of real objects described). When the "imagination" mode is active, the model can even be trained to wrap its inferences in special tokens (e.g., [object]), providing a built-in confidence signal for the user.
Conclusion
HallE-Control represents a significant step toward more reliable and controllable LMMs. By acknowledging that hallucination is not always "bad"—for example, in robotics where inferring occluded objects is useful—the authors provide a framework to manage it rather than just eliminate it. While currently focused on object existence, this "control knob" approach offers a promising path for addressing attribute and relationship hallucinations in future multimodal research.
Summary updated