Abstract
Visual hallucination (VH) occurs when a multi-modal large language model (MLLM) imagines incorrect details about an image during visual question answering. Existing studies often rely on standard image datasets, leading to a biased understanding of MLLM performance due to limited diversity in hallucination instances. In this work, we introduce VHTest, a tool designed to generate a diverse set of VH instances. VHTest identifies initial failures in existing datasets, generates descriptive text for these failure modes, and utilizes text-to-image generative models (e.g., DALL-E 3) to create new, challenging images. We present a benchmark of 1,200 VH instances across eight modes. Our findings reveal that state-of-the-art models like GPT-4V frequently hallucinate on these instances, but fine-tuning on our benchmark can significantly mitigate these errors without compromising general performance.
The Problem: Hidden Blind Spots in MLLMs
Multi-modal Large Language Models (MLLMs) have shown remarkable capabilities, yet they remain prone to Visual Hallucinations (VH)—instances where the model "sees" objects that do not exist or mischaracterizes attributes like color, size, and position.
Previous benchmarking efforts relied on existing datasets like COCO. However, these datasets suffer from limited diversity and potential data contamination, as they are likely part of the models' original training sets. Consequently, a model might appear robust simply by recalling training data rather than accurately processing visual input. VHTest addresses this by generating novel, "adversarial" images specifically designed to trigger these failures.
Methodology: The VHTest Pipeline
The researchers developed a three-step pipeline to systematically uncover and expand upon model weaknesses:
- Identifying Initial Failures: The process begins by finding image pairs where the CLIP vision encoder (common in MLLMs) perceives high similarity, but the DINO v2 encoder identifies significant differences. These discrepancies highlight potential blind spots in the vision-language interface.
- Textual Mode Generation: Successful hallucination instances are analyzed by an MLLM to describe the nature of the error and suggest parameters for generating similar challenging cases.
- Generative Expansion: These descriptions guide a text-to-image model (DALL-E 3) to produce high-quality, synthetic images. Human annotators then verify these images and create ground-truth question-answer pairs to finalize the benchmark.
The VHTest Benchmark
The resulting benchmark consists of 1,200 instances categorized into eight distinct hallucination modes: existence, shape, color, orientation, OCR, size, position, and counting.
By moving beyond standard datasets, VHTest provides a more rigorous assessment of model robustness. The inclusion of modes like "orientation" and "relative size" reveals that models may correctly identify an object's existence while failing completely to understand its spatial context.
Key Results and Insights
The evaluation of leading models on VHTest revealed significant reliability gaps:
- Performance Deficit: On open-ended questions, GPT-4V achieved only 38.3% accuracy, while LLaVA-1.5 and MiniGPT-v2 scored 22.9% and 7.5%, respectively.
- Challenging Modes: Orientation and OCR were the most difficult categories, with average accuracies across models dropping as low as 14% for orientation.
- Mitigation through Fine-tuning: A key finding is that these synthetic instances are not just for testing; they are effective for training. Fine-tuning LLaVA-1.5 on a subset of VHTest data improved its accuracy in the "position" mode from 33.3% to 53.3% without degrading performance on other standard benchmarks.
Conclusion
VHTest demonstrates that adversarial generative construction is a powerful tool for diagnosing and fixing the reliability gaps in MLLMs. While the current process requires human verification, it paves the way for more autonomous and diverse safety testing in multi-modal AI development. The benchmark is publicly available to encourage further research into robust visual perception.
Summary updated