The Curse of Multi-Modalities: Evaluating Hallucinations of Large Multimodal Models across Language, Visual, and Audio

Sicong Leng, Yun Xing, Zesen Cheng, Yang Zhou, Hang Zhang, Xin Li, Deli Zhao, Shijian Lu, Chunyan Miao, Lidong BingSubmission to ICLR 2025 · Rejected

TL;DR

While Large Multimodal Models (LMMs) increasingly combine text, vision, and audio, adding more modalities exacerbates hallucinations. This paper introduces The Curse of Multi-Modalities (CMM), a diagnostic benchmark showing that tri-modal hallucinations are primarily driven by over-reliance on unimodal priors (especially visual and linguistic dominance) and spurious inter-modality correlations learned during pre-training.

What Problem Does This Paper Address?

Most research on multimodal hallucination focuses strictly on static vision-language pairs (e.g., detecting non-existent objects in photos). However, real-world frontier models now process video and audio streams simultaneously. Accommodating these extra streams significantly complicates cross-modal alignment and fusion.

This paper investigates how and why hallucinations occur across the triad of language, vision, and audio, establishing the first systematic study of hallucinations in tri-modal contexts.

Methods

The authors trace hallucinations to two fundamental mechanisms:

  1. Over-reliance on unimodal priors: Models default to dominant patterns in a single modality—such as linguistic common sense (Language Dominance), visual cues overpowering audio (Visual Dominance), or auditory cues overpowering video (Audio Dominance).
  2. Spurious inter-modality correlations: Models infer absent events or objects purely from high co-occurrence statistics in pre-training data (e.g., hallucinating a dog bark whenever a dog is visible or whimpering).

To validate these phenomena, the authors designed CMM, a benchmark framing evaluation as object- and event-level binary probing questions ("yes"/"no"):

  • Perception Accuracy (PA): $\frac{\text{\# correct 'yes'}}{\text{\# ground-truth 'yes'}}$ (ability to detect real items).
  • Hallucination Resistance (HR): $\frac{\text{\# correct 'no'}}{\text{\# ground-truth 'no'}}$ (ability to reject absent items).

They also conducted perturbation experiments (e.g., blurring video frames or injecting audio noise) to verify that degrading a dominant modality forces the model to shift reliance to the other senses.

Datasets

The authors curated CMM, containing 1,200 samples (audio-only, video-only, and video-audio pairs) and 2,400 balanced probing queries evenly split between existent ("yes") and non-existent ("no") targets across 6 subcategories.

  • Data Sources:
  • Video clips and visual annotations: Derived from WebVid-10M and YouTube (for anti-commonsense cases like flying fish or black bananas).
  • Audio and video-audio pairs: Sourced from AudioCaps and Auto-ACD.
  • Relation to Prior Benchmarks: Unlike earlier benchmarks (e.g., POPE, HallusionBench, AVHBench) that either evaluate only image-text pairs or omit inter-modality correlations, CMM provides fine-grained object- and event-level probing across all three modalities simultaneously.

Key Insights and Ideas

  • Vision steamrolls audio: Models are much more susceptible to Visual Dominance than Audio Dominance. Seeing a hammer leads models to hallucinate the sound of hammering, even when the audio track contains only birds chirping.
  • Audio-Language associations are fragile: While years of research have tackled visual-language hallucinations, audio-language integration remains a blind spot. Models routinely hallucinate expected audio events because of pre-training co-occurrence.
  • Scaling helps language bias, not data co-occurrence: Scaling the LLM backbone significantly improves resistance against Language Dominance (common-sense bias), but barely impacts spurious correlations caused by dataset-wide co-occurrences.
  • Chain-of-Thought (CoT) is a double-edged sword: Asking models to "think step-by-step" often harms perceptual accuracy or triggers catastrophic drops in hallucination resistance, as longer text outputs give language priors more opportunity to override sensory grounding.

Key Results

The authors evaluated leading proprietary models (Gemini-1.5/2.0, Reka-Core) and open-source models (VideoLLaMA 2, Qwen2.5-Omni, FAVOR, GroundingGPT):

  • Severe Audio-Language (AL) vulnerabilities: Even frontier models stumbled on AL spurious correlations. Gemini-1.5-Pro reached 94.0% PA but plummeted to 14.5% HR. Open-source GroundingGPT achieved 100% PA but 0.0% HR on AL correlations due to an affirmative ("yes") bias.
  • Visual vs. Audio Dominance: On Gemini-1.5-Flash, Hallucination Resistance was 86.5% for Audio Dominance, but collapsed to 36.5% for Visual Dominance.
  • Model Behavior Styles: Some models exhibited extreme overconfidence (GroundingGPT answered "yes" almost universally, resulting in 14.3% overall HR), whereas safety-aligned models like Reka-Core preferred cautious rejection, achieving 80.9% overall HR at the expense of lower PA (63.7%).

Limitations and Future Work

  • Binary Probing vs. Free-form Tasks: CMM relies on short yes/no queries. While diagnostic, this does not capture complex, multi-step open-ended generation.
  • Affirmative Response Biases: Open-source multimodal models frequently over-predict affirmative answers, which can complicate pure perceptual evaluation.
  • Future Directions: The authors emphasize the need for datasets with balanced modality representation, context-aware dynamic cross-modal fusion, and explicit visual/audio fact-checking mechanisms during decoding.

Summary updated

Abstract

Recent advancements in large multimodal models (LMMs) have significantly enhanced performance across diverse tasks, with ongoing efforts to further integrate additional modalities such as video and audio. However, most existing LMMs remain vulnerable to hallucinations, the discrepancy between the factual multimodal input and the generated textual output, which has limited their applicability in various real-world scenarios. This paper presents the first systematic investigation of hallucinations in LMMs involving the three most common modalities: language, visual, and audio. Our study reveals two key contributors to hallucinations: overreliance on unimodal priors and spurious inter-modality correlations. To address these challenges, we introduce the benchmark \textit{The Curse of Multi-Modalities} (\textbf{CMM}), which comprehensively evaluates hallucinations in LMMs, providing a detailed analysis of their underlying issues. Our findings highlight key vulnerabilities, including imbalances in modality integration and biases from training data, underscoring the need for balanced cross-modal learning and enhanced hallucination mitigation strategies. Based on our observations and findings, we suggest potential research directions that could enhance the reliability of LMMs. We will make our code and data publicly available.
BibTeX
@misc{leng2024curse,
  author = {Sicong Leng and Yun Xing and Zesen Cheng and Yang Zhou and Hang Zhang and Xin Li and Deli Zhao and Shijian Lu and Chunyan Miao and Lidong Bing},
  title = {{The Curse of Multi-Modalities: Evaluating Hallucinations of Large Multimodal Models across Language, Visual, and Audio}},
  year = {2024},
  eprint = {2410.12787},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2410.12787},
}

Loading related content...

Figure