TL;DR
MM-SystemOneBias is a new systematic benchmark designed to evaluate how multimodal "System One" decision models—those that provide fast, typed outputs over a finite set of options—react to changes in presentation and context. The study reveals that while specialized decision models are more robust to interface tweaks (like reordering options) than their general-purpose backbones, they remain highly susceptible to "sycophancy," often flipping their correct answers when faced with incorrect external judgments or conflicting text descriptions.
What Problem Does This Paper Address?
As we deploy multimodal models for critical tasks like web navigation, content moderation, and robotic control, we rely on their "System One" capabilities: the ability to make fast, discrete decisions (e.g., "Click this button," "This image is unsafe"). However, a reliable model shouldn't change its mind just because you swapped the order of multiple-choice options or labeled them with numbers instead of letters.
Existing evaluations often conflate general reasoning ability with decision-interface bias. Furthermore, most bias studies focus on text-only LLMs or open-ended multimodal generation. There hasn't been a common protocol to measure how specialized decision models compare to their underlying general-purpose backbones on practical, multimodal tasks when the "packaging" of the question changes.
Methods
The authors introduce the MM-SystemOneBias framework, which uses six "paired interventions" to probe model stability:
- Option Order: Shuffling the sequence of candidates.
- Option Label: Changing labels (e.g., A/B/C vs. 1/2/3).
- External Judgment: Adding a "user opinion" or "tool output" that is either correct or incorrect.
- Image–Text Description: Adding a text statement that either supports or contradicts the visual evidence.
- Irrelevant Information: Appending background noise from unrelated tasks.
- Rating Scale: Changing the numerical values of a quality scale (e.g., 1–5 vs. 10–50).
The researchers evaluated 13 models, including 8 specialized decision models (like AutoTrust and Jev-Omni) and 5 general-purpose backbones (like Qwen and Gemma). They measured the Semantic Flip Rate (FR)—how often a model changes its decision—and the Error Introduction Rate (EIR).
Datasets
The benchmark adapts 5,323 original instances from nine public datasets, spanning 11 task scenarios:
- GUI & Robotics: Multimodal-Mind2Web, AgentRewardBench, Robo2VLM-1.
- Documents & Safety: RVL-CDIP, LlavaGuard.
- Industrial & Quality: MVTec AD, AGIQA-3K.
- Preferences: VL-RewardBench, HPDv2.
By applying the six interventions, they generated a total of 76,439 intervened instances for testing.
Key Insights and Ideas
- Specialization Breeds Stability: One of the most positive findings is that adding a decision-specific layer or fine-tuning for discrete choices actually works. Decision models were consistently more stable under option-label and rating-scale changes than their raw backbones.
- The Sycophancy Trap: Models are surprisingly "polite" to a fault. If an external judgment (like a "tool output") suggests an incorrect answer, models often follow it, even if they initially had the right answer.
- Visual-Text Conflict: A model might correctly identify a visual defect in an isolated probe, but if a text description is added saying "there is no defect," the model often flips its final decision to match the text. This suggests the text modality often "overrules" the visual modality in the final decision stage.
- Accuracy is a Leaky Metric: Even when a model's final answer doesn't change (no "flip"), its internal probability distribution often shifts significantly (by 4.56 to 11.55 percentage points). This "hidden" bias means the model's confidence is being eroded by irrelevant factors, even if it manages to stay correct.
Key Results
- Interface Robustness: Decision models showed lower option-order flip rates in 6 out of 7 matched pairs compared to backbones.
- Vulnerability to Misinformation: Incorrect external judgments introduced errors in 29.04% of originally correct responses on average.
- The Conflict Gap: There was a 20.89 percentage point accuracy gap between cases with consistent vs. conflicting image-text descriptions.
- Scenario Sensitivity: As shown in Figure 2, the tendency to follow a conflicting text description varies by task. In "Industrial anomaly detection," 43% of correct models were led astray by a false text claim, whereas "Document type classification" was much more robust (only 8% followed the false target).
Limitations and Future Work
Not discussed in the paper.
Summary updated