Magnifier Prompt: Tackling Multimodal Hallucination via Extremely Simple Instructions

Yuhan Fu, Ruobing Xie, Jiazhen Liu, Bangxiang Lan, Xingwu Sun, Zhanhui Kang, Xirong Li

TL;DR

This paper introduces "Magnifier Prompt" (MagPrompt), a simple, training-free prompting strategy to significantly reduce hallucinations in multimodal large language models (MLLMs). By instructing MLLMs to prioritize image content and resolve conflicts in favor of visual information, MagPrompt achieves comparable or superior performance to complex methods across various datasets and models, including closed-source ones like GPT-4o.

What Problem Does This Paper Address?

Multimodal Large Language Models (MLLMs), which combine visual understanding with language generation, are powerful but suffer from a significant problem: hallucinations. These are instances where the model generates information that is not present in the input image or contradicts it, often due to the model's inherent bias towards its pre-existing textual knowledge over visual cues. This tendency to "hallucinate" severely limits their reliability and practical application. Existing methods to combat this often involve complex architectural changes or specialized decoding strategies, which can be difficult to implement, especially for proprietary, closed-source models. The paper aims to find a simpler, more accessible solution.

Methods

The core contribution is the Magnifier Prompt (MagPrompt), a novel prompting technique designed to guide MLLMs towards more accurate visual grounding. MagPrompt is built upon two fundamental principles:

  1. Image Focus: MLLMs should prioritize processing and relying on the visual information provided in the image.
  2. Image Priority: In cases of conflict between the image content and the model's internal knowledge (common sense), the image should take precedence.

The MagPrompt itself is a template that wraps the user's original query, embedding these principles as explicit instructions. For example, a prompt might instruct the model to "Carefully observe the image and provide an answer based on the content of the image. In cases where the image conflicts with your knowledge or common sense, prioritize the image." This method is training-free, meaning it doesn't require any model fine-tuning, making it highly versatile.

Datasets

The paper evaluates MagPrompt on several established datasets designed to test multimodal hallucination:

  • POPE (Perception-Oriented Prompting Evaluation): Used across various splits (random, popular, adversarial) to assess performance.
  • PhD (Prompted Visual Hallucination Evaluation Dataset): Evaluated on both neutral and misleading questioning modes.
  • AMBER: Focuses on discriminative tasks.
  • HallusionBench: Features manually collected images and more complex questions.

The authors also mention experiments with InstructBLIP on the POPE adversarial split, noting its performance limitations, which they attribute to its shorter instruction-tuning data.

Key Insights and Ideas

  • Simplicity is Key: The most striking insight is that extremely simple, instruction-based prompts can be remarkably effective in mitigating complex issues like multimodal hallucinations. This challenges the notion that only intricate architectural or algorithmic changes can solve such problems.
  • Instruction Following Matters: The success of MagPrompt hinges on the MLLM's ability to follow instructions. Models with stronger instruction-following capabilities (like GPT-4o and Gemini Pro) benefit more, highlighting a crucial aspect for future MLLM development.
  • Beyond F1p: The paper argues that relying solely on the positive F1 score (F1p) can be misleading, as it favors models biased towards "yes" answers. They emphasize the importance of considering the negative F1 score (F1N) and macro F1 for a more balanced evaluation of hallucination mitigation.
  • Universality: MagPrompt's effectiveness across diverse models, including closed-source ones, demonstrates its broad applicability and potential as a general-purpose tool for improving MLLM reliability.

Key Results

MagPrompt consistently improved hallucination mitigation across various models and datasets:

  • LLaVA-1.5 (7B/13B): MagPrompt outperformed VCD on the POPE adversarial split, showing significant improvements in F1N and macro F1. For example, on LLaVA-1.5-13B, MagPrompt achieved a macro F1 of 84.5, compared to VCD's 81.9.
  • GPT-4o and Gemini Pro: MagPrompt demonstrated comparable or better performance than baseline methods, showcasing its effectiveness even on state-of-the-art closed-source models.
  • POPE Adversarial Split: MagPrompt consistently improved macro F1 scores, with notable gains like 5.22% on LLaVA-1.5-7B and 13.21% on mPLUG-Owl.
  • PhD Dataset: MagPrompt showed improvements in both neutral and misleading modes for GPT-4V, indicating its robustness.

Limitations and Future Work

The authors acknowledge several limitations:

  • Prompt Engineering: While MagPrompt is simple, they suggest that more advanced prompting techniques like Chain-of-Thought (CoT) reasoning could potentially yield even better results, which they leave for future research.
  • Dataset Scope: The evaluated datasets, while diverse, represent a limited range of tasks and scenarios compared to the vast possibilities in real-world applications. Further deployment and evaluation in more varied contexts are needed.
  • Hallucinations Persist: Despite MagPrompt's effectiveness, MLLMs still exhibit hallucinations. Fully resolving this complex issue remains a significant challenge.

Future work could explore integrating MagPrompt with other techniques, expanding its application to more complex scenarios, and further investigating the interplay between instruction-following capabilities and hallucination mitigation.

Summary updated

Abstract

Hallucinations in multimodal large language models (MLLMs) hinder their practical applications. To address this, we propose a Magnifier Prompt (MagPrompt), a simple yet effective method to tackle hallucinations in MLLMs via extremely simple instructions. MagPrompt is based on the following two key principles, which guide the design of various effective prompts, demonstrating robustness: (1) MLLMs should focus more on the image. (2) When there are conflicts between the image and the model's inner knowledge, MLLMs should prioritize the image. MagPrompt is training-free and can be applied to open-source and closed-source models, such as GPT-4o and Gemini-pro. It performs well across many datasets and its effectiveness is comparable or even better than more complex methods like VCD. Furthermore, our prompt design principles and experimental analyses provide valuable insights into multimodal hallucination.
BibTeX
@misc{fu2024magnifier,
  author = {Yuhan Fu and Ruobing Xie and Jiazhen Liu and Bangxiang Lan and Xingwu Sun and Zhanhui Kang and Xirong Li},
  title = {{Magnifier Prompt: Tackling Multimodal Hallucination via Extremely Simple Instructions}},
  year = {2024},
  eprint = {2410.11701},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2410.11701},
}
Figure