Abstract
Large Language Models (LLMs) are increasingly utilized as automated judges for Natural Language Generation (NLG) tasks. However, this study identifies a significant "criteria confusion" where LLMs struggle to isolate specific quality dimensions such as fluency, coherence, and faithfulness. Using a framework of 18 targeted perturbation attacks and a hierarchical taxonomy of 11 criteria, we demonstrate that LLMs often exhibit a "halo effect," penalizing unrelated aspects due to localized errors. Our findings suggest that current LLM-based evaluation lacks the fine-grained diagnostic capability of human annotators.
Introduction
In the evaluation of Natural Language Generation (NLG), the goal is to assess multiple distinct dimensions of quality, including grammatical correctness, logical flow, and factual accuracy. Traditional metrics like BLEU and ROUGE often fail to capture these nuances, leading to the adoption of LLMs (e.g., GPT-4) as "human-like" evaluators.
A critical concern in this transition is the "halo effect"—a phenomenon where an evaluator's overall impression of a text biases their scores for specific, independent criteria. If an LLM cannot distinguish between a summary that is ungrammatical and one that is factually incorrect, its utility as a diagnostic tool is severely limited. This paper investigates the extent of this confusion and the factors that exacerbate it.
Methodology
The researchers developed a systematic framework to test the discriminative power of LLM evaluators.
Hierarchical Taxonomy
To clarify the relationships between different evaluation aspects, the authors organized 11 common criteria into a hierarchical structure. This taxonomy distinguishes between "Readability" (internal text quality) and "Adequacy" (relationship to source material).
Perturbation Attacks
The core of the methodology involves 18 "perturbation attacks" designed to degrade specific quality aspects while leaving others intact. For instance:
- Inversion: Swapping word order to damage Fluency while maintaining Faithfulness.
- Sentence Exchange: Reordering sentences to disrupt Coherence without affecting Grammaticality.
- Conflicting Fact: Altering specific details to compromise Faithfulness while preserving Readability.
Experimental Setup
The study spanned four tasks: News Summarization (CNN/DM), Dialogue Summarization (SAMSum), Paraphrase Generation, and Table-to-Text Generation. Evaluations were conducted using various prompt styles, ranging from simple terms to detailed rubrics.
Results and Discussion
The experiments revealed a stark contrast between human and LLM evaluation patterns.
The Correlation Gap
A primary indicator of criteria confusion is the high correlation between scores for different aspects. As shown in the heatmap below, LLM-generated scores for Fluency, Coherence, Relevance, and Consistency are significantly more correlated than those provided by human annotators, suggesting the model fails to isolate specific qualities.
Key Findings
- Grammar Penalty Leakage: LLMs frequently penalized Coherence and Simplicity for texts containing minor spelling or grammatical errors, even when the logic remained sound.
- Contradiction Sensitivity: When a text contradicted the source material, LLMs lowered scores across all categories, including Readability, which should theoretically be independent of the source content.
- Terminology Dominance: LLMs often relied on their internal pre-trained definitions of terms like "Fluency" rather than following the specific instructions provided in the evaluation prompt.
- The GPT-4 Paradox: While GPT-4 is more sensitive to quality drops than GPT-3.5, it paradoxically showed higher levels of confusion in "invariance tests"—the ability to keep scores stable for aspects not targeted by a perturbation.
Conclusion
While LLMs offer a scalable alternative to human evaluation, they currently suffer from a lack of "criteria isolation." They tend to provide a holistic "vibe" check rather than a precise multi-dimensional assessment. Human annotators remain superior in following hierarchical systems, achieving a 94.4% success rate in isolating criteria compared to the significant "leakage" observed in LLMs. Future research must focus on de-biasing techniques and architectural changes to ensure LLMs can function as objective, fine-grained judges.
Summary updated