Abstract
Evaluating Multimodal Large Language Models (MLLMs) remains a significant challenge due to the limitations of traditional automated metrics. This paper introduces MLLM-as-a-Judge, a comprehensive benchmark designed to evaluate whether MLLMs can effectively serve as autonomous evaluators for vision-language tasks. By analyzing performance across various evaluation settings and domains, the study reveals that while top-tier models like GPT-4V show promise, significant hurdles such as response length bias and hallucinations persist.
Introduction
Traditional automated metrics like BLEU or CIDEr often fail to capture the nuance, creativity, and reasoning required in modern multimodal applications, such as explaining complex charts or solving geometry problems. While the "LLM-as-a-Judge" paradigm has become a staple for evaluating text-only models, its applicability to the multimodal domain—where visual context adds a layer of complexity—remained unexplored. This research investigates whether MLLMs can effectively serve as judges and how closely their evaluations align with human preferences.
Methodology
The researchers developed a robust three-step framework to evaluate MLLM judging capabilities:
- Image-Instruction Pair Collection: Gathering diverse prompts and images across multiple domains.
- Response Collection: Generating answers from six mainstream MLLMs, including GPT-4V, Gemini, and LLaVA.
- Judgment & Comparison: Using a "Judge MLLM" to evaluate these responses across three distinct settings: Scoring Evaluation (1–5 rating), Pair Comparison (choosing the better of two), and Batch Ranking (ordering four responses).
Additionally, the study explored an "Analyze-then-Judge" approach (multimodal Chain-of-Thought) and a "Vision Expert" setting, where the judge receives a detailed text description of the image instead of raw pixels.
Benchmark Composition
The benchmark is built upon a curated collection of 4,414 image-instruction pairs sourced from 14 diverse datasets covering 10 domains, including reasoning (MathVista), perception (MS COCO), and infographics (ChartQA). The authors also released two specialized subsets:
- MLLM-AS-A-JUDGE-HQ: Responses with high human-model agreement.
- MLLM-AS-A-JUDGE-HARD: A "stress test" containing instances of hallucinations and inconsistent judgments to challenge future models.
Key Insights and Analysis
Several critical biases and behaviors were identified during the evaluation process:
- The "High-Score" Bias: Most models (excluding GPT-4V) tend to cluster their scores around 4 or 5, struggling to penalize poor-quality or incorrect responses effectively.
- Verbosity Bias: MLLMs exhibit a strong preference for longer responses. Increasing the length of a response without adding new information significantly increases the likelihood of a higher score from a judge model.
- The CoT Paradox: While Chain-of-Thought (CoT) reasoning is generally beneficial, multi-step CoT occasionally decreased judging performance. Models would sometimes hallucinate details in their analysis and use those false premises to justify final judgments.
- Description vs. Vision: Surprisingly, models often performed better as judges when provided with a textual description of the image rather than the image itself, suggesting current MLLMs still struggle to bridge the gap between raw visual perception and logical evaluation.
Experimental Results
- GPT-4V Performance: GPT-4V emerged as the gold standard, consistently outperforming other models and showing the highest alignment with human annotations (Pearson similarity of 0.490 in scoring).
- Optimal Evaluation Setting: Pair Comparison proved to be the most reliable setting, with GPT-4V achieving approximately 79% agreement with human judges.
- Scaling and Bias: Larger models (e.g., LLaVA-1.6-34b) showed clear improvements in consistency. However, an "egocentric bias" was noted, where models like GPT-4V occasionally favored their own response style.
Conclusion
While MLLMs show potential as autonomous evaluators, they are not yet fully reliable. Hallucinations and inherent biases remain significant hurdles. Future work should focus on multimodal RLHF/DPO to better align MLLMs with human judgment, developing techniques to mitigate length and position biases, and exploring hybrid human-in-the-loop systems for complex cases.
Summary updated