Introduction
As Natural Language Processing (NLP) models grow in complexity, traditional automated metrics like BLEU and ROUGE are increasingly inadequate for capturing nuanced qualities such as coherence, plausibility, and helpfulness. This has led to the rapid adoption of "LLM-as-a-judge," where powerful models like GPT-4 are used to evaluate the outputs of other systems. However, the validity of this approach remains under-explored. This paper introduces JUDGE-BENCH, a comprehensive evaluation suite designed to determine whether Large Language Models (LLMs) can truly replace human experts across diverse linguistic tasks.
The JUDGE-BENCH Framework
JUDGE-BENCH aggregates 20 existing NLP datasets, totaling over 70,000 instances. The benchmark covers two primary types of evaluation:
- Categorical Tasks: Including grammatical acceptability (CoLA), reasoning validity, and toxicity/safety (ToxicChat, DICES).
- Graded Tasks: Including dialogue quality (Topical Chat), machine translation (WMT), and summarization (SummEval).
The study evaluated 11 prominent LLMs, ranging from proprietary giants like GPT-4o and Gemini-1.5 to leading open-weight models like Llama-3.1 and Mixtral-8x22B. Researchers primarily used the original instructions provided to human annotators to test the models' ability to follow standard rubrics.
Key Insights: The Expert Gap and Bias
The most significant finding of the study is the "Expert Gap." While LLMs show a promising ability to align with non-expert, crowdsourced workers, they struggle significantly to match the rigorous standards of domain experts.
As illustrated in the figure above, LLMs tend to mirror the varied scores of non-experts in tasks like the Switchboard Telephone Corpus. However, in expert-led tasks like WMT 2023, a stark contrast emerges: while an expert might assign a nuanced score (e.g., 59), LLMs frequently over-estimate quality with much higher, less discriminating scores (e.g., 90-96). This suggests that LLMs may rely on surface-level patterns that satisfy casual readers but fail to capture the deeper linguistic precision required by experts.
Furthermore, the study identified a Self-Generation Bias. Models are notably better at judging human-written text than machine-generated content. This "blind spot" is particularly concerning given that LLMs are most frequently used to evaluate other machine-generated outputs in real-world applications.
Performance and Results
- Top Performers: GPT-4o, Llama-3.1-70B, and Mixtral-8x22B emerged as the most reliable judges. In factual consistency tasks (QAGS), GPT-4o achieved a correlation of 0.72, nearing the human upper bound of 0.74.
- The Open-Weight Rise: The gap between proprietary models and open-weight models is narrowing, which bodes well for the reproducibility of AI evaluation.
- Safety Refusals: A critical limitation was noted in safety and medical domains, where internal guardrails often caused LLMs to "refuse" to provide a judgment, leading to poor alignment in sensitive areas.
Conclusion
While LLMs are powerful evaluators, they are not yet "drop-in" replacements for human experts. Their inconsistency across tasks and their struggle with machine-generated text suggest that researchers should remain cautious. The authors recommend validating LLM judges against a small set of human expert annotations before relying on them for full-scale evaluations.
Summary updated