Mitigating Hallucination in Large Vision-Language Models through Aligning Attention Distribution to Information Flow

Jianfei Zhao, Feng Zhang, Xin Sun, Chong FengEMNLP 2025

TL;DR

Large Vision-Language Models (LVLMs) often "hallucinate" by generating content inconsistent with the visual input. This paper proposes SEVI, a training-free method that aligns the model's attention distribution with the natural flow of visual information into semantic representations, significantly reducing hallucinations. SEVI achieves this by identifying and propagating attention patterns from "core semantic heads" through a novel two-stage optimization process.

What Problem Does This Paper Address?

Large Vision-Language Models (LVLMs) are powerful tools that integrate visual understanding with language generation, enabling tasks like image captioning. However, a significant challenge is their tendency to "hallucinate" – generating descriptions that are not supported by the visual input. This unreliability severely limits their practical application. Existing methods often focus on improving training data or manipulating language priors, but this paper argues that a key issue lies in the misalignment between how LVLMs process visual information internally and how they distribute their attention. Specifically, the authors observe that while visual information is progressively absorbed into semantic representations as it flows through the model, the model's attention mechanism doesn't adequately reflect this flow, leading to hallucinations.

Methods

The core of this paper is the Semantic-Enhanced Visual Interpretation (SEVI) method, a training-free approach designed to align the LVLM's attention distribution with the information flow.

Here's how it works:

  1. Information Flow Analysis: The authors first analyze how visual information propagates through LVLMs. They use a technique similar to "Attention Rollout" to visualize how visual features are integrated into semantic representations across different layers. They find that visual information is largely absorbed into semantic representations in later layers.
  2. Identifying "Core Semantic Heads": Based on this analysis, they hypothesize that attention should be focused on these semantic representations. They identify specific "attention heads" within the model that disproportionately attend to these core semantic representations. These are termed "core semantic heads."
  3. Two-Stage Attention Optimization: SEVI then employs a two-stage optimization process to guide the model's attention:
  4. Stage 1 (Global Semantic Heads): The model's attention is first guided by "global semantic heads" (heads that attend to semantic representations but not necessarily the "core" ones). This stage aims to align the overall attention distribution with the visual information flow.
  5. Stage 2 (Core Semantic Heads): Subsequently, the "core semantic heads" are used to guide the global semantic heads. This stage encourages the model to focus more intensely on the most critical information, thereby suppressing hallucinations.
  6. Weighted Smoothing: A weighted smoothing mechanism is incorporated to ensure stable performance improvements during this optimization.
  7. Mitigating Language Priors (CICD): To prevent the model from becoming overly reliant on language priors (which can also contribute to hallucinations), SEVI integrates CICD (Cross-Image Contrastive Decoding), a technique that uses image consistency to identify and remove detrimental language priors.

Datasets

The paper evaluates its method on three widely adopted image captioning benchmarks:

  • CHAIR: Focuses on detecting object hallucinations.
  • DetailCaps: A fine-grained benchmark with ground-truth detail captions.
  • AMBER: Features high-quality images with manually annotated objects for detailed hallucination evaluation.

The authors tested their method on five different LVLMs: InstructBLIP, LLaVA-1.5, LLaVA-Next, Qwen2-VL-Instruct, and Qwen2.5-VL-Instruct.

Key Insights and Ideas

  • Visual information flow is key: The paper's central insight is that hallucinations arise not just from poor visual understanding but from a misalignment between how visual information flows into semantic representations and where the model directs its attention.
  • Semantic representations are crucial anchors: As visual information is integrated into semantic representations, these become critical points for understanding. Focusing attention here is more effective than just trying to boost raw visual attention.
  • Attention heads are specialized: The authors leverage the fact that different attention heads naturally specialize. They identify the "expert" heads (core semantic heads) and use their learned patterns to guide the "novice" heads.
  • A trade-off exists: The paper acknowledges a fundamental trade-off between reducing hallucinations (becoming more conservative) and preserving detailed descriptions (being more descriptive). SEVI offers a way to control this balance.

Key Results

SEVI demonstrates significant effectiveness in reducing hallucinations across various LVLMs and benchmarks.

  • Hallucination Reduction: SEVI consistently achieves lower hallucination rates compared to baseline methods. For instance, on CHAIR, SEVI (Focused mode) achieves Cs scores as low as 14.8 (LLaVA-1.5) and 13.4 (LLaVA-1.5-13B), significantly outperforming regular decoding.
  • Balanced vs. Focused Modes: The paper introduces two modes:
  • Focused mode: Aggressively minimizes hallucinations, leading to the lowest hallucination rates but sometimes a slight drop in recall (descriptive detail).
  • Balanced mode: Strikes a compromise, maintaining low hallucination rates while preserving more details.
  • Generalization: The method shows strong generalization capabilities, performing well across five different LVLMs.
  • Model Size Impact: The paper notes that SEVI shows even greater improvement on larger models (e.g., 13B vs. 7B), suggesting it effectively harnesses their latent capacity.

Limitations and Future Work

  • Trade-off between Hallucinations and Details: The authors explicitly state that while SEVI reduces hallucinations, the "Focused mode" can lead to overly conservative outputs, potentially overlooking fine-grained details. This is reflected in a slight decrease in recall. They introduce hyperparameters to allow users to manually adjust this trade-off.
  • Flow Mechanism vs. Attention: SEVI optimizes attention distribution based on the underlying information flow but does not directly enhance the flow mechanism itself.
  • Future Work: The authors plan to explore more efficient strategies for visual information propagation in future work. They also note that as a training-free approach, SEVI is inherently limited by the performance ceiling of the base LVLM.

Summary updated

BibTeX
@inproceedings{zhao2025mitigating,
  author = {Jianfei Zhao and Feng Zhang and Xin Sun and Chong Feng},
  title = {{Mitigating Hallucination in Large Vision-Language Models through Aligning Attention Distribution to Information Flow}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
}
Figure