TL;DR
This paper critically examines the claim that Large Language Models (LLMs) can fully replace human relevance assessments in information retrieval. The authors argue that while LLMs show high correlation with human judgments, theoretical concerns and practical limitations, such as the risk of circular evaluation and inherent biases, mean they are not yet a viable substitute for human evaluation.
What Problem Does This Paper Address?
The field of information retrieval (IR) relies heavily on relevance assessments to evaluate the effectiveness of search systems. Traditionally, these assessments are performed by humans, a time-consuming and expensive process. Recent advancements in Large Language Models (LLMs) have led to systems like UMBRELA that can automatically generate relevance judgments, showing high correlation with human assessments. This has sparked a debate: can LLMs now replace humans entirely for relevance assessment, particularly in benchmark evaluations like those conducted by TREC? This paper directly challenges this notion, arguing that the current LLM-based approaches, while promising, are not yet robust enough to supplant human judgment.
Methods
The authors critically analyze the findings of Upadhyay et al. [11], which claimed LLM-based relevance assessments could replace human ones. They employ several methods to support their counter-argument:
- Re-analysis of Existing Data: They re-examine the TREC 2024 RAG track data used by Upadhyay et al. [11], focusing on the correlation between LLM (UMBRELA) and human judgments, particularly for top-performing systems.
- Systematic Discrepancy Analysis: They investigate specific instances where LLM and human assessments diverge significantly, highlighting potential weaknesses in LLM evaluations.
- Simulating Circularity: They simulate a scenario where LLM-based assessments are used both for evaluating systems and as part of the systems themselves (e.g., for re-ranking). This explores the consequences of "circular evaluation," where the evaluation metric becomes part of the system being evaluated.
- Empirical Demonstration of Subversion: They describe a deliberately crafted system (run
uwc1) designed to exploit automatic evaluation metrics, demonstrating how such systems can achieve artificially inflated scores.
Datasets
The primary dataset used for evaluation is the TREC 2024 RAG track data. This dataset consists of 301 queries over the MS MARCO Segment V2.1 collection, with participating systems producing ranked lists of 100 passages per query. The authors note that they were granted limited access to the data by Upadhyay et al. [11] to confirm factual statements.
Key Insights and Ideas
- Correlation is Not Equivalence: While LLM-based assessments correlate highly with human judgments, this correlation weakens significantly among top-performing systems, making it difficult to discern true progress.
- The "Narcissism" of LLMs: LLMs tend to favor outputs generated by other LLMs, leading to a potential bias where LLM-based rankers are unfairly favored by LLM-based evaluators.
- Vulnerability to Manipulation: Automatic evaluation metrics, especially those based on LLMs, are susceptible to manipulation. Systems can be designed to exploit these metrics without necessarily improving true retrieval quality.
- The Peril of Circular Evaluation: When LLM-based assessment tools are also used within the systems being evaluated (e.g., for re-ranking), it creates a feedback loop that invalidates the evaluation process, leading to inflated scores and distorted rankings.
- LLMs are Not People: The fundamental argument remains that IR systems are designed to serve human needs, and therefore, their evaluation must ultimately reflect human judgment and preferences, which LLMs, by their nature, cannot fully replicate.
Key Results
- Weakening Correlation at the Top: Kendall's τ drops from 0.89 overall to 0.51 among the top 20 systems and 0.56 among the top 15 systems when comparing LLM (UMBRELA) and manual assessments. This indicates a significant misalignment in ranking top systems.
- Subverted System Performance: The deliberately crafted
uwc1system ranked 5th under automatic assessment but only 28th under manual assessment, demonstrating the potential for LLM-based evaluations to be gamed. - Circularity Inflates Scores: In a simulated circular evaluation, manual NDCG scores for top systems ranged from 0.68 to 0.72, while the same systems achieved scores between 0.95 and 1.00 under LLM-based evaluation, indicating substantial score inflation.
Limitations and Future Work
- Preliminary Nature of Upadhyay et al. [11]: The authors emphasize that the conclusions of Upadhyay et al. [11] are based on a preliminary analysis, and their own work provides a critical counterpoint.
- Need for Critical Consideration: The paper calls for more critical consideration of LLM-based evaluation claims, especially given the authority of the researchers involved.
- Addressing Theoretical Concerns: The authors reiterate the theoretical concerns raised by Faggioli et al. [6] (e.g., unknown biases, overfitting, potential degradation of future LLM performance) which remain unaddressed by the proponents of LLM replacement.
- Safeguards Against Feedback Loops: Future work needs to focus on developing safeguards to prevent feedback loops in LLM-based evaluation pipelines, particularly for reusable test collections.
- Defining a True Gold Standard: The paper strongly advocates for human judgments as the true gold standard for relevance assessment, emphasizing the need for LLMs to align with these judgments rather than replace them.
Summary updated