TL;DR
This paper argues that Large Language Models (LLMs) cannot yet replace human judgment for relevance assessment in information retrieval, despite recent claims suggesting otherwise. The authors highlight practical issues like the ease of gaming LLM-based evaluations and theoretical concerns such as LLM "narcissism" and the risk of circular evaluation loops.
What Problem Does This Paper Address?
The field of Information Retrieval (IR) relies heavily on relevance assessments to evaluate the performance of search systems. Traditionally, human judges meticulously assess whether documents retrieved by a search system are relevant to a given query. Recently, there's been a surge of interest in using LLMs to automate this process, with some studies suggesting LLM-based assessments are comparable to human ones. A recent paper by Upadhyay et al. [11] even claims that LLM-based assessments, specifically using the UMBRELA system, can fully replace human judgments in TREC-style evaluations. This paper critically examines that claim, arguing that this replacement is premature due to practical and theoretical limitations.
Methods
The authors critically analyze the findings of Upadhyay et al. [11] by:
- Re-examining correlation metrics: They investigate the reported correlation between LLM and human judgments, specifically looking at how this correlation changes when focusing on top-performing systems.
- Empirical Demonstration of Subversion: They describe a deliberately crafted submission (run
uwc1) designed to exploit automatic evaluation metrics, demonstrating how easily LLM-based evaluations can be manipulated to achieve artificially high scores without genuine improvement in retrieval quality. - Simulation of Circularity: They simulate a scenario where LLM-based assessments are used both for evaluation and as a re-ranking mechanism within the systems being evaluated. This explores the potential consequences of a feedback loop where LLMs judge outputs generated by LLMs.
Datasets
The paper primarily uses data from the TREC 2024 RAG track. This track involved participants executing 301 queries over the MS MARCO Segment V2.1 collection, producing ranked lists of 100 passages per query. The authors also refer to data from the TREC 2021 Deep Learning track and the MS MARCO collection.
Key Insights and Ideas
- Correlation weakens at the top: While LLM and human judgments show high correlation overall, this alignment significantly degrades when focusing on the top-performing systems, making it difficult to discern genuine progress.
- Automatic evaluations are easily gamed: Systems can be designed to exploit the known mechanisms of LLM-based evaluators, leading to inflated scores that don't reflect true retrieval quality.
- LLM "narcissism" is a real concern: LLMs tend to favor outputs generated by other LLMs, creating a bias that undermines objective evaluation.
- Circularity is a looming threat: When LLMs are used for both evaluation and system components (like re-ranking), it creates a feedback loop where the evaluation metric becomes part of the system, rendering the evaluation invalid.
- LLMs are not people: The fundamental purpose of IR systems is to serve human needs, and therefore, their evaluation must ultimately reflect human judgment and preferences, which LLMs currently cannot fully replicate.
Key Results
- The correlation (Kendall's τ) between manual and UMBRELA assessments drops significantly for top-performing systems: from 0.85 for the top 60 systems to 0.51 for the top 20 systems.
- A deliberately manipulated run (
uwc1) ranked 5th under automatic assessment but only 28th under manual assessment, demonstrating the ease of subversion. - In a simulated circular evaluation scenario, the correlation (Kendall's τ) dropped sharply to 0.63 for the top 60 systems, and even became negative (-0.40) for the top 5 systems, indicating substantial score inflation.
Limitations and Future Work
Limitations: * The authors acknowledge that the TREC 2024 RAG track data used for some analyses was preliminary and that a full public data release was not available at the time of writing. * The simulation of circularity is a hypothetical scenario, though the authors argue it's a plausible future outcome.
Future Work: The paper strongly advocates for continued reliance on human judgments for relevance assessment. It implies that before LLMs can be considered viable replacements, significant theoretical challenges, such as LLM narcissism and the risk of circular evaluation, must be addressed. Future work should focus on developing safeguards against feedback loops and ensuring that LLM-based evaluations genuinely reflect human needs and preferences.
Summary updated