Introduction
Multimodal Large Language Models (MLLMs) have demonstrated remarkable capabilities in complex visual reasoning. However, recent evaluations suggest they may struggle with fundamental perceptual tasks when visual data conflicts with textual semantics. The "What Color Is It?" benchmark highlights a specific vulnerability: "loyalty hallucination." This phenomenon occurs when a model correctly perceives a visual attribute but ultimately defers to a contradictory textual label present within the image.
The Challenge of Semantic Leakage
While MLLMs excel at high-level understanding, they are susceptible to semantic leakage. This happens when surrounding context—such as a label or descriptive text—interferes with the model's objective visual judgment. Previous research primarily examined conflicts between external text prompts and images. This study shifts the focus to internal image modality conflicts, embedding contradictory text directly into the visual field to test if models can resolve discrepancies between pixel-level data and semantic meaning.
Methodology: A Systematic Framework
To measure single-modality visual hallucination, the researchers developed a framework consisting of three core components within a single image:
- The Background: A visual field that can be neutral or colored.
- The Question: A query (e.g., "What color is it?") that can itself serve as a colored distractor.
- The Text: The target object, which may be a color-related word (distractor) or a neutral word.
The study also utilizes Chain-of-Thought (CoT) analysis. By examining the internal "thought" process of reasoning-heavy models like GLM4v-Thinking and Qwen3-VL, researchers can determine if a model initially identifies the correct color but later "corrects" its answer to match the text's meaning—a clear indicator of linguistic bias.
Dataset Composition
The "What Color Is It" dataset comprises three subsets, each containing four types of samples ranging from simple to complex:
- Color Subset: The primary interference group where text content is a color name different from its actual ink color (e.g., the word "navy" written in pink).
- Simple Subset: A control group using color-neutral words (e.g., "February") to establish baseline color recognition performance.
- Mask Subset: A variation where characters are partially masked (e.g., "o**nge") to test if models "auto-complete" text and then hallucinate based on that completion.
Each subset includes 400 images, totaling 1,200 test cases used to evaluate 12 state-of-the-art models, including GPT-4o and Gemini 2.0 Flash.
Key Insights: Loyalty Hallucination and Textual Dominance
The most significant finding is "Loyalty Hallucination." CoT logs reveal that models often "see" the truth but "choose" the lie. For instance, a model might correctly note "I see blue text" in its internal reasoning but then provide an incorrect final answer because the word itself says "red." This suggests a "blind faith in text" stemming from training on massive text-heavy corpora, where linguistic priors often override visual signals.
Furthermore, models fall into an "Auto-Complete Trap." In the Mask subset, models use internal language logic to fill in blanks. Once a model decides a masked word is "orange," it frequently ignores the fact that the actual pixels are blue.
As visual complexity increases—such as in Type 3 and Type 4 tasks involving multiple colors—accuracy drops significantly. The increased cognitive load appears to make models even more reliant on easy-to-parse text distractors.
Quantitative Results
Evaluation across 12 models revealed stark performance gaps:
- High Failure Rates: On the Color subset (Type 1), models like InternVL3.5-14B and Qwen2.5-VL-7B exhibited 0% accuracy, failing to ignore the text distractor in every instance.
- Reasoning Model Performance: While advanced models like GPT-4o performed better on simple tasks (55% accuracy on Type 1), their performance plummeted to just 9% on Type 4 tasks.
- The Gemini Paradox: Gemini 2.0 Flash showed strong baseline recognition but was highly susceptible to "Type II Hallucination," where the color of the question text interfered with the answer, resulting in a 58% hallucination rate in specific setups.
- Control Group Success: Most models performed well on the Simple subset, confirming that the failure is not due to an inability to perceive color, but an inability to prioritize visual data over conflicting semantics.
Conclusion and Future Directions
The "What Color Is It?" benchmark exposes a critical weakness in current MLLMs: a deep-seated linguistic bias that can override visual reality. While the current dataset uses synthetic images, it provides a vital standard for testing multimodal integration. Future development must focus on improving visual grounding mechanisms to ensure models can explicitly isolate visual features from semantic labels, moving closer to robust, human-like visual intelligence.
Summary updated