VidHal: Benchmarking Temporal Hallucinations in Vision LLMs

Wey Yeh Choong, Yangyang Guo, Mohan S. Kankanhalli

Abstract

Vision Large Language Models (VLLMs) are widely acknowledged to be prone to hallucinations. Existing research addressing this problem has primarily been confined to image inputs, with limited exploration of video-based hallucinations. Furthermore, current evaluation methods fail to capture nuanced errors in generated responses, which are often exacerbated by the rich spatiotemporal dynamics of videos. To address this, we introduce VidHal, a benchmark specially designed to evaluate video-based hallucinations in VLLMs. VidHal is constructed by bootstrapping video instances across a wide range of common temporal aspects. A defining feature of our benchmark lies in the careful creation of captions which represent varying levels of hallucination associated with each video. To enable fine-grained evaluation, we propose a novel caption ordering task requiring VLLMs to rank captions by hallucinatory extent. We conduct extensive experiments on VidHal and comprehensively evaluate a broad selection of models. Our results uncover significant limitations in existing VLLMs regarding hallucination generation. Through our benchmark, we aim to inspire further research on 1) holistic understanding of VLLM capabilities, particularly regarding hallucination, and 2) extensive development of advanced VLLMs to alleviate this problem.

Introduction

While Vision Large Language Models (VLLMs) have shown remarkable progress, they frequently "hallucinate"—generating plausible-sounding but factually incorrect descriptions. Most existing evaluation frameworks focus on static images, identifying errors like incorrect object colors or non-existent items. However, videos introduce the complex dimension of time, where a model might correctly identify an action but fail to grasp its direction (e.g., catching vs. throwing) or sequence.

We introduce VidHal, a benchmark designed to expose these "temporal hallucinations." By moving beyond simple multiple-choice questions to a novel caption ordering task, VidHal demonstrates that even state-of-the-art models like GPT-4o struggle with nuanced temporal reasoning, often relying on static image "priors" rather than true video dynamics.

Methodology

The VidHal benchmark consists of 1,000 video instances curated from public datasets like TempCompass, Perception Test, MVBench, and AutoEval-Video. The construction pipeline follows a four-stage process:

  1. Temporal Aspect Selection: Focus on five key dimensions: Attribute, Object, Action, Direction, and Event Order.
  2. Video Instance Collection: Sourcing videos that clearly exhibit these temporal dynamics.
  3. Anchor Caption Generation: Using GPT-4o to generate a ground-truth "Anchor Caption" based on video metadata and QA pairs.
  4. Hallucinatory Caption Generation: Generating two additional captions representing "Moderate" and "High" levels of error by modifying specific temporal aspects.
The VidHal construction pipeline, illustrating the transition from temporal aspect selection to the generation of varying levels of hallucinatory captions.

To evaluate models, we employ standard Multiple Choice Question Answering (MCQA) and a core innovation: Caption Ordering. In this task, models must rank three captions from most accurate to most hallucinatory. Performance is measured using Relative NDCG, rewarding models for correct relative rankings even if the absolute order is imperfect. Human validation confirmed the benchmark's quality with an 87% agreement rate.

Experimental Results

Our evaluation of a broad selection of VLLMs reveals several key insights:

  • The "Image Prior" Trap: Many VLLMs exhibit a heavy reliance on single frames. When tested on a single salient frame versus the full video, many models produced identical, often incorrect, responses, suggesting they "guess" temporal dynamics from static snapshots.
  • Difficulty with Direction and Order: While models perform reasonably well on Action and Object recognition, they struggle significantly with Direction (e.g., clockwise vs. counter-clockwise) and Event Order.
  • Proprietary vs. Open-Source: GPT-4o emerged as the top performer (MCQA: 0.772, Relative NDCG: 0.826). Among open-source models, VideoLLaMA2 (72B) and LLaVA-NeXT-Video (32B) were competitive but still trailed significantly.
Radar chart comparing the performance of various VLLMs across five temporal aspects, highlighting the relative strengths and weaknesses in handling direction, action, object, attribute, and order.

The results indicate a "hallucination misalignment": models often fail to distinguish between the ground truth and a "moderate" hallucination, frequently ranking slightly incorrect captions as more accurate than the truth.

Conclusion and Future Work

VidHal exposes the "image-centric" bias of current VLLMs. While proprietary models lead the field, there is a significant gap in fine-grained temporal reasoning across all models. Future development should focus on pretraining strategies that emphasize temporal logic over simple object identification. While VidHal provides a diagnostic foundation with 1,000 videos, scaling to longer, more complex sequences remains a critical challenge for the community.

Summary updated

Abstract

Vision Large Language Models (VLLMs) are widely acknowledged to be prone to hallucinations. Existing research addressing this problem has primarily been confined to image inputs, with limited exploration of video-based hallucinations. Furthermore, current evaluation methods fail to capture nuanced errors in generated responses, which are often exacerbated by the rich spatiotemporal dynamics of videos. To address this, we introduce VidHal, a benchmark specially designed to evaluate video-based hallucinations in VLLMs. VidHal is constructed by bootstrapping video instances across a wide range of common temporal aspects. A defining feature of our benchmark lies in the careful creation of captions which represent varying levels of hallucination associated with each video. To enable fine-grained evaluation, we propose a novel caption ordering task requiring VLLMs to rank captions by hallucinatory extent. We conduct extensive experiments on VidHal and comprehensively evaluate a broad selection of models. Our results uncover significant limitations in existing VLLMs regarding hallucination generation. Through our benchmark, we aim to inspire further research on 1) holistic understanding of VLLM capabilities, particularly regarding hallucination, and 2) extensive development of advanced VLLMs to alleviate this problem.
BibTeX
@misc{choong2024vidhal,
  author = {Wey Yeh Choong and Yangyang Guo and Mohan S. Kankanhalli},
  title = {{VidHal: Benchmarking Temporal Hallucinations in Vision LLMs}},
  year = {2024},
  eprint = {2411.16771},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2411.16771},
}
Figure