Seeing Clearly by Layer Two: Enhancing Attention Heads to Alleviate Hallucination in LVLMs

Xiaofeng Zhang, Yihao Quan, Chaochen Gu, Chen Shen, Xiaosong Yuan, Shaotian Yan, Hao Cheng, Kaijie Wu, Jieping YearXiv 2024 · DOI · arXiv

Large Vision-Language Models (LVLMs) such as LLaVA and InternVL have demonstrated remarkable capabilities in multimodal tasks. However, they remain prone to "hallucinations"—generating descriptions of objects or attributes not present in the visual input. While existing solutions often involve computationally expensive retraining or complex decoding strategies, recent research suggests a simpler, training-free intervention at the model's early layers.

The Problem: Sparse Vision Sinks

Hallucinations often occur when an LVLM prioritizes linguistic priors over actual visual evidence. For instance, if a model identifies a "dining table," it might reflexively predict the presence of a "person" regardless of the image content.

The paper "Seeing Clearly by Layer Two" identifies a structural correlate for this behavior: "vision sinks." These are specific columns in the attention matrix where image tokens capture significant attention. The researchers found that when early layers—specifically Layer 2—exhibit "sparse" attention patterns (few vision sinks), the model is significantly more likely to hallucinate.

Comparison of sparse vs. dense vision sink heads in Layer 2 and their impact on object grounding.

Methodology: Enhancing Attention Heads (EAH)

To address this, the authors propose Enhancing Attention Heads (EAH), a "plug-and-play" method that requires no additional training. The approach follows three steps:

  1. Identify Vision Sinks: Define columns in the attention matrix where the average attention score for image tokens exceeds a specific threshold ($\beta$).
  2. Locate the "Dense" Head: In Layer 2, identify the attention head with the highest proportion of these vision sinks.
  3. Broadcast the Map: Extract the attention matrix from this "densest" head and copy (broadcast) it to all other heads within the same layer.

By anchoring all attention heads in Layer 2 to the most image-focused pattern, the model maintains a stronger visual "grounding" before proceeding to deeper reasoning layers.

Key Insights and Results

The core finding is that the information flow for image tokens is heavily concentrated in the first two layers. If the model fails to establish dense vision sinks by Layer 2, the visual anchor is lost. Statistical analysis shows a direct correlation: non-hallucinated tokens typically activate a much larger number of dense heads than hallucinated ones.

Evidence linking hallucinated tokens to a lower proportion of dense vision sink heads in Layer 2 across multiple models.

EAH was evaluated across several benchmarks with impressive results:

  • Hallucination Reduction: On the CHAIR dataset, EAH reduced the sentence-level hallucination score ($CHAIR_S$) for LLaVA-1.5 from 47.0 to 36.4.
  • Object Grounding: On the POPE benchmark, it achieved an F1 score of 85.7, surpassing other training-free methods like VCD and OPERA.
  • Generalization: The method proved effective across seven different LVLMs (including InternVL and Shikra) and even showed benefits for pure LLMs like Llama-3.

Conclusion and Future Work

EAH offers a highly efficient way to mitigate hallucinations by simply "aligning" the model's internal eyesight at a critical early stage. While it can lead to a slight decrease in recall (the model mentions fewer objects overall), its training-free nature and minimal overhead make it a powerful tool for improving LVLM reliability. Future work may explore how the optimal intervention layer shifts as model architectures become deeper and more complex.

Summary updated

Abstract

The hallucination problem in multimodal large language models (MLLMs) remains a common issue. Although image tokens occupy a majority of the input sequence of MLLMs, there is limited research to explore the relationship between image tokens and hallucinations. In this paper, we analyze the distribution of attention scores for image tokens across each layer and head of the model, revealing an intriguing and common phenomenon: most hallucinations are closely linked to the pattern of attention sinks in the self-attention matrix of image tokens, where shallow layers exhibit dense attention sinks and deeper layers show sparse attention sinks. We further analyze the attention heads of different layers and find that heads with high-density attention sink in the image part play a positive role in alleviating hallucinations. In this paper, we propose a training-free method named \textcolor{red}{\textbf{E}}nhancing \textcolor{red}{\textbf{A}}ttention \textcolor{red}{\textbf{H}}eads (EAH), an approach designed to enhance the convergence of image tokens attention sinks in the shallow layers. EAH identifies the attention head that shows the vision sink in a shallow layer and extracts its attention matrix. This attention map is then broadcast to other heads in the layer, thereby strengthening the layer to pay more attention to the image itself. With extensive experiments, EAH shows significant hallucination-mitigating performance on different MLLMs and metrics, proving its effectiveness and generality.
BibTeX
@misc{zhang2024seeing,
  author = {Xiaofeng Zhang and Yihao Quan and Chaochen Gu and Chen Shen and Xiaosong Yuan and Shaotian Yan and Hao Cheng and Kaijie Wu and Jieping Ye},
  title = {{Seeing Clearly by Layer Two: Enhancing Attention Heads to Alleviate Hallucination in LVLMs}},
  year = {2024},
  eprint = {2411.09968},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2411.09968},
}
Figure