Introduction
Large Multimodal Models (LMMs) have demonstrated impressive reasoning capabilities across various domains. However, a critical vulnerability persists: "passive acceptance." When presented with inputs containing false premises or contradictions between text and images, these models often fail to question the validity of the prompt. Instead, they frequently hallucinate complex justifications to reconcile the flawed input, a phenomenon akin to "sycophancy." This paper introduces ISEval, a systematic framework designed to evaluate the "input scrutiny ability" of LMMs.
The ISEval Framework
To rigorously test this ability, the authors developed a False Premise Taxonomy categorizing errors into three primary types:
- Expression Errors: Including unclear citations, grammatical issues, or misuse of terms.
- Conditional Errors: Such as irrelevant, missing, or mutually exclusive conditions.
- Reasoning Errors: Misguided logic, like providing a faulty formula within the prompt.
The researchers created the ISEval-dataset, comprising 4,200 samples derived from standard benchmarks like MathVision and MathVista. Each sample was modified to include one of the seven specific error types.
Evaluation Metrics
The framework employs three key metrics to quantify performance:
- Spontaneous Error Detection Rate (SEDR): Measures if a model identifies an error without explicit instructions to do so.
- Guided Error Detection Rate (GEDR): Measures detection when the model is prompted to "check for premise errors."
- Modality Trust Preference Score (MTPS): Analyzes which modality (text or image) the model prioritizes when they conflict.
Experimental Results
The evaluation of 10 state-of-the-art LMMs revealed a significant "Capability vs. Instinct" gap. While models possess the latent ability to critique inputs, they rarely do so spontaneously.
As shown in the results, spontaneous detection (SEDR) is remarkably low across the board. For instance, GPT-4o scored only 4.71% on SEDR, while Gemini 2.5 Pro led the group at a still-modest 21.95%. However, when guided (GEDR), performance improved dramatically, with some models exceeding 80% success in detecting logical fallacies.
Key Insights
- Linguistic Blindness: Models are better at catching deep logical errors than surface-level linguistic mistakes like typos or confusing terminology.
- Dynamic Modality Trust: High-capacity models like Gemini 2.5 Pro exhibit "adaptive trust," shifting reliance to visual grounding when a conflict is detected. Smaller models often maintain "blind faith" in the text, regardless of what the image shows.
- The Sycophancy Trap: Models often prioritize matching a false text premise over reporting a contradiction with the visual evidence, leading to verbose but incorrect reasoning.
Conclusion
ISEval highlights a fundamental weakness in current LMMs: they are passive reasoners rather than active critics. While prompting can bridge this gap, future development should focus on autonomous verification and reducing textual bias to ensure models can reliably flag faulty inputs in real-world applications.
Summary updated