Judging LLMs on a Simplex

Patrick Vossler, Fan Xia, Yifan Mai, Jean Feng

TL;DR

This paper introduces a novel geometric "simplex" perspective to analyze the increasingly common practice of using LLMs to judge other LLMs. By mapping judges and candidates onto a probability simplex, the authors prove that while binary (2-level) scoring is theoretically robust, multi-level scoring is mathematically "non-identifiable" without strong prior beliefs about judge quality. To solve this, they propose a Bayesian framework that accounts for both random sampling noise and systematic judge bias, providing more reliable model rankings and better uncertainty quantification than standard methods.

What Problem Does This Paper Address?

As LLMs generate more complex, free-form text (like mathematical proofs or clinical reasoning), traditional metrics like BLEU or exact-match accuracy fail. The industry has pivoted to "LLM-as-a-judge," where a powerful model (like GPT-4o) grades the outputs of other models.

However, this practice rests on a shaky foundation: it assumes the judge is perfectly accurate or that its errors magically cancel out. Current uncertainty tools, like bootstrapping, only account for aleatoric uncertainty (random noise from a small sample size). They completely ignore epistemic uncertainty—the fact that we don't actually know the "true" quality of the judge. If the judge is biased or inconsistent, our rankings might be fundamentally wrong, no matter how many samples we collect.

Methods

The authors represent the "LLM-as-a-judge" problem through the lens of an $(M-1)$-dimensional probability simplex.

  • Geometric Mapping: For an $M$-level scoring system (e.g., a 3-point scale of "Incorrect," "Partial," "Correct"), every candidate model and judge can be represented as a point or a set of vertices in this geometric space.
  • Identifiability Theory: They use this geometry to provide "visual proofs" of when a ranking is mathematically recoverable. They show a "phase transition" where 2-level systems are easy to identify, but 3+ level systems require additional constraints.
  • Bayesian Framework: They develop a hierarchical model that:
  • Uses random effects to relax the "constancy" assumption (the idea that a judge performs identically across all candidates).
  • Employs a weight propagation graph to encode priors on judge quality.
  • Conducts sensitivity analyses by systematically varying hyperparameters to see if candidate rankings "flip" under different assumptions about judge bias.

Datasets

The framework was evaluated across five diverse benchmarks representing different levels of "verifiability":

  1. GPQA & MMLU Pro: Verifiable tasks with 2-level scores (Correct/Incorrect).
  2. MTBench & SummEval: Multi-level tasks originally judged by humans on Likert scales (5-point or 10-point).
  3. Omni-MATH: A "semi-verifiable" task involving complex olympiad-level math, using a 3-level scale (Correct, Partial, Incorrect).

Key Insights and Ideas

  • The "Folk Wisdom" is Mathematically True: Researchers often prefer binary rubrics over nuanced scales. This paper provides the formal proof: in a 2-level system, you can recover the true ranking even with a mediocre judge. In a 3+ level system, the geometry allows for multiple "equally valid" judge configurations that produce opposite rankings.
  • Epistemic Uncertainty is the Real Boss: Standard error bars (like those from bootstrapping) are often too narrow because they ignore judge quality. The authors show that accounting for epistemic uncertainty leads to much wider—but more "honest"—confidence intervals.
  • Robustness is Dataset-Specific: Some benchmarks are "easy" to rank because the models are far apart in the simplex (e.g., MTBench). Others are "fragile" because models cluster in the center (e.g., Omni-MATH), meaning a tiny shift in judge bias can completely reorder the leaderboard.

Key Results

  • Superior Coverage: The Bayesian method achieved significantly higher "coverage rates" (the probability that the true ranking falls within the estimated interval). On the GPQA dataset, the Bayesian method hit 77.8% coverage, compared to just 50% for bootstrapping and 44.4% for Bradley-Terry models.
  • Stability in Rankings: Sensitivity analysis showed that for most datasets, rankings remain stable even when the "perfect judge" assumption is relaxed. However, MMLU Pro and Omni-MATH showed high sensitivity, warning researchers to be cautious with those specific leaderboards.
  • Self-Preference Correction: The model successfully identified and corrected for "self-preference bias" (where an LLM judge gives higher scores to models from its own family), a known issue in LLM evaluation.

Limitations and Future Work

  • Computational Overhead: Running Bayesian posterior inference via Markov Chain Monte Carlo (MCMC) is more computationally expensive than simple score averaging.
  • The Need for Priors: While the framework quantifies uncertainty, it still requires the user to define "minimally-informative" priors. If the judge is truly adversarial (worse than random), the system still fails.
  • Future Directions: The authors suggest extending this geometric perspective to more complex evaluation structures, such as pairwise comparisons that go beyond the standard Bradley-Terry assumptions, and exploring how to automatically "tune" the strength of the priors based on small subsets of gold-standard human data.

Summary updated

Abstract

Automated evaluation of free-form outputs from large language models (LLMs) is challenging because many distinct answers can be equally valid. A common practice is to use LLMs themselves as judges, but the theoretical properties of this approach are not yet well understood. We show that a geometric framework that represents both judges and candidates as points on a probability simplex can provide helpful insight on what is or is not identifiable using LLM judges. Our theoretical analysis uncovers a "phase transition" in ranking identifiability: for binary scoring systems, true rankings are identifiable even with weak judges under mild assumptions, while rankings become non-identifiable for three or more scoring levels even with infinite data, absent additional prior knowledge. This non-identifiability highlights how uncertainty in rankings stems from not only aleatoric uncertainty (i.e., inherent stochasticity in the data) but also epistemic uncertainty regarding which assumptions hold, an aspect that has received limited attention until now. To integrate both types of uncertainty, we use Bayesian inference to encode assumptions as priors and conduct sensitivity analysis of ranking estimates and credible intervals. Empirical evaluations across multiple benchmarks demonstrate that Bayesian inference yields more accurate rankings and substantially improves coverage rates. These results underscore the importance of taking a more holistic approach to uncertainty quantification when using LLMs as judges.
BibTeX
@inproceedings{vossler2026judging,
  author = {Patrick Vossler and Fan Xia and Yifan Mai and Jean Feng},
  title = {{Judging LLMs on a Simplex}},
  booktitle = {International Conference on Artificial Intelligence and Statistics},
  year = {2026},
  url = {https://openreview.net/forum?id=HxmC9vLZE3},
}
Figure