Meta Ranking: Less Capable Language Models are Capable for Single Response Judgement

Zijun Liu, Boqun Kou, Peng Li, Ming Yan, Ji Zhang, Fei Huang, Yang Liu

Abstract

Despite the strong performance of large language models (LLMs) across a wide range of tasks, they still have reliability issues. Previous studies indicate that strong LLMs like GPT-4-turbo excel in evaluating the reliability of responses from LLMs, but face efficiency and local deployment issues. Thus, to enable weak LLMs to effectively assess the reliability of LLM responses, we propose a novel cross-query-comparison-based method called Meta Ranking (MR). Unlike previous few-shot methods that solely based on in-context learning capabilities in LLMs, MR assesses reliability by pairwisely ranking the target query-response pair with multiple reference query-response pairs. We found that MR is highly effective in error detection for LLM responses, where weak LLMs, such as Phi-2, could surpass strong baselines like GPT-3.5-turbo, requiring only five reference samples and significantly improving efficiency. We further demonstrate that MR can enhance strong LLMs' performance in two practical applications: model cascading and instruction tuning. In model cascading, we combine open- and closed-source LLMs to achieve performance comparable to GPT-4-turbo with lower costs. In instruction tuning, we use MR for iterative training data filtering, significantly reducing data processing time and enabling LLaMA-7B and Phi-2 to surpass Alpaca-13B with fewer training tokens. These results underscore the high potential of MR in both efficiency and effectiveness.

Introduction

As Large Language Models (LLMs) are increasingly integrated into critical workflows, the need for reliable "judges"—models that can evaluate the correctness of an AI's output—has become paramount. Currently, this role is predominantly filled by "strong" models like GPT-4. However, these models are often closed-source, expensive, and slow, making them unsuitable for local deployment or high-throughput applications.

Smaller, "weak" models (typically under 10B parameters) offer a potential solution for efficient, local evaluation. Unfortunately, these models are notoriously poor judges. When tasked with direct evaluation via standard few-shot prompting, they frequently exhibit over-confidence (labeling incorrect answers as correct) and permutation sensitivity (changing their judgment based on the order of examples in the prompt).

Methodology: Meta Ranking (MR)

To address these limitations, we introduce Meta Ranking (MR). The core insight of MR is that while weak models struggle with absolute judgments, they are remarkably capable of relative comparisons. Instead of asking a model if a specific response is correct, MR asks it to rank the reliability of a "target" query-response pair against a set of "reference" pairs with known reliability.

Comparison of Meta Ranking with previous methods and its conceptual workflow.

The MR process involves three main steps:

  1. Reference Selection: A small set (e.g., 5) of query-response pairs with pre-determined reliability scores is used.
  2. Cross-Query Comparison: The weak LLM performs pairwise ranking between the target pair and each reference pair.
  3. Aggregation: The results are aggregated through a voting mechanism. If the target is ranked higher than a "correct" reference, it receives an upvote; if lower than an "incorrect" reference, it receives a downvote.

This approach is model-agnostic and bypasses the structural weaknesses of small models by converting a complex absolute judgment task into a series of simpler relative comparisons.

Experimental Results

We evaluated MR across diverse benchmarks, including reasoning (MMLU, CMMLU), translation (FLORES-200), and instruction tuning datasets.

Precision of Meta Ranking (MR) compared to standard prompting (P(T)) and other baselines across various LLMs.

The results demonstrate a significant performance leap for weak models. For instance, on the MMLU dataset, the 2.7B Phi-2 model using MR achieved a precision of 0.77, more than doubling its performance with standard prompting and even surpassing the performance of GPT-3.5-turbo. This suggests that MR effectively "levels the playing field," allowing small, efficient models to perform at the level of much larger counterparts in evaluation tasks.

Practical Applications

Beyond simple error detection, MR enables two high-impact applications:

1. Model Cascading

By using a weak model with MR as a "router," we can create a cost-effective hybrid system. Easy queries are handled locally by the weak model, while only those flagged as "unreliable" are sent to a strong, expensive model like GPT-4. This approach achieved performance comparable to GPT-4-turbo while using less than half the API tokens.

2. Instruction Tuning

MR can be used for iterative training data filtering. By filtering out low-quality data after each training epoch, we implemented a form of "curriculum learning." This allowed a LLaMA-7B model to reach an MT-Bench score of 4.69, outperforming the Alpaca-13B model trained on the full, unfiltered dataset.

Conclusion and Future Work

Meta Ranking demonstrates that low-capacity models can be highly effective judges when the task is reframed from absolute evaluation to relative comparison. While MR shows some performance degradation in low-resource languages like Japanese due to the underlying model's limitations, its success in English and Chinese is compelling. Future work will explore applying MR to strong models to further push the boundaries of automated evaluation and integrating MR directly into the LLM alignment process.

Summary updated

Abstract

Despite the strong performance of large language models (LLMs) across a wide range of tasks, they still have reliability issues. Previous studies indicate that strong LLMs like GPT-4-turbo excel in evaluating the reliability of responses from LLMs, but face efficiency and local deployment issues. Thus, to enable weak LLMs to effectively assess the reliability of LLM responses, we propose a novel cross-query-comparison-based method called $\textit{Meta Ranking}$ (MR). Unlike previous few-shot methods that solely based on in-context learning capabilities in LLMs, MR assesses reliability by pairwisely ranking the target query-response pair with multiple reference query-response pairs. We found that MR is highly effective in error detection for LLM responses, where weak LLMs, such as Phi-2, could surpass strong baselines like GPT-3.5-turbo, requiring only five reference samples and significantly improving efficiency. We further demonstrate that MR can enhance strong LLMs' performance in two practical applications: model cascading and instruction tuning. In model cascading, we combine open- and closed-source LLMs to achieve performance comparable to GPT-4-turbo with lower costs. In instruction tuning, we use MR for iterative training data filtering, significantly reducing data processing time and enabling LLaMA-7B and Phi-2 to surpass Alpaca-13B with fewer training tokens. These results underscore the high potential of MR in both efficiency and effectiveness.
BibTeX
@misc{liu2024meta,
  author = {Zijun Liu and Boqun Kou and Peng Li and Ming Yan and Ji Zhang and Fei Huang and Yang Liu},
  title = {{Meta Ranking: Less Capable Language Models are Capable for Single Response Judgement}},
  year = {2024},
  eprint = {2402.12146},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2402.12146},
}
Figure