Can Large Multimodal Models Actively Recognize Faulty Inputs? A Systematic Evaluation Framework of Their Input Scrutiny Ability

Haiqi Yang, Jinzhe Li, Gengxu Li, Yi Chang, Yuan WuarXiv 2025 · DOI · arXiv

Introduction

Large Multimodal Models (LMMs) have demonstrated impressive reasoning capabilities across various domains. However, a critical vulnerability persists: "passive acceptance." When presented with inputs containing false premises or contradictions between text and images, these models often fail to question the validity of the prompt. Instead, they frequently hallucinate complex justifications to reconcile the flawed input, a phenomenon akin to "sycophancy." This paper introduces ISEval, a systematic framework designed to evaluate the "input scrutiny ability" of LMMs.

The ISEval Framework

To rigorously test this ability, the authors developed a False Premise Taxonomy categorizing errors into three primary types:

  1. Expression Errors: Including unclear citations, grammatical issues, or misuse of terms.
  2. Conditional Errors: Such as irrelevant, missing, or mutually exclusive conditions.
  3. Reasoning Errors: Misguided logic, like providing a faulty formula within the prompt.
Figure 1: The ISEval framework illustrates the difference between models with and without input scrutiny ability, alongside the taxonomy of false premises used for evaluation.

The researchers created the ISEval-dataset, comprising 4,200 samples derived from standard benchmarks like MathVision and MathVista. Each sample was modified to include one of the seven specific error types.

Evaluation Metrics

The framework employs three key metrics to quantify performance:

  • Spontaneous Error Detection Rate (SEDR): Measures if a model identifies an error without explicit instructions to do so.
  • Guided Error Detection Rate (GEDR): Measures detection when the model is prompted to "check for premise errors."
  • Modality Trust Preference Score (MTPS): Analyzes which modality (text or image) the model prioritizes when they conflict.

Experimental Results

The evaluation of 10 state-of-the-art LMMs revealed a significant "Capability vs. Instinct" gap. While models possess the latent ability to critique inputs, they rarely do so spontaneously.

Figure 3: Detailed performance metrics (SEDR and GEDR) across various LMMs and error categories. Note the significant jump in performance when models are explicitly guided to look for errors.

As shown in the results, spontaneous detection (SEDR) is remarkably low across the board. For instance, GPT-4o scored only 4.71% on SEDR, while Gemini 2.5 Pro led the group at a still-modest 21.95%. However, when guided (GEDR), performance improved dramatically, with some models exceeding 80% success in detecting logical fallacies.

Key Insights

  • Linguistic Blindness: Models are better at catching deep logical errors than surface-level linguistic mistakes like typos or confusing terminology.
  • Dynamic Modality Trust: High-capacity models like Gemini 2.5 Pro exhibit "adaptive trust," shifting reliance to visual grounding when a conflict is detected. Smaller models often maintain "blind faith" in the text, regardless of what the image shows.
  • The Sycophancy Trap: Models often prioritize matching a false text premise over reporting a contradiction with the visual evidence, leading to verbose but incorrect reasoning.

Conclusion

ISEval highlights a fundamental weakness in current LMMs: they are passive reasoners rather than active critics. While prompting can bridge this gap, future development should focus on autonomous verification and reducing textual bias to ensure models can reliably flag faulty inputs in real-world applications.

Summary updated

Abstract

Large Multimodal Models (LMMs) have witnessed remarkable growth, showcasing formidable capabilities in handling intricate multimodal tasks with exceptional performance. Recent research has underscored the inclination of large language models to passively accept defective inputs, often resulting in futile reasoning on invalid prompts. However, the same critical question of whether LMMs can actively detect and scrutinize erroneous inputs still remains unexplored. To address this gap, we introduce the Input Scrutiny Ability Evaluation Framework (ISEval), which encompasses seven categories of flawed premises and three evaluation metrics. Our extensive evaluation of ten advanced LMMs has identified key findings. Most models struggle to actively detect flawed textual premises without guidance, which reflects a strong reliance on explicit prompts for premise error identification. Error type affects performance: models excel at identifying logical fallacies but struggle with surface-level linguistic errors and certain conditional flaws. Modality trust varies-Gemini 2.5 pro and Claude Sonnet 4 balance visual and textual info, while aya-vision-8b over-rely on text in conflicts. These insights underscore the urgent need to enhance LMMs' proactive verification of input validity and shed novel insights into mitigating the problem. The code is available at https://github.com/MLGroupJLU/LMM_ISEval.
BibTeX
@misc{yang2025can,
  author = {Haiqi Yang and Jinzhe Li and Gengxu Li and Yi Chang and Yuan Wu},
  title = {{Can Large Multimodal Models Actively Recognize Faulty Inputs? A Systematic Evaluation Framework of Their Input Scrutiny Ability}},
  year = {2025},
  eprint = {2508.04017},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2508.04017},
}
Figure