Towards Provably Unbiased LLM Judges via Bias-Bounded Evaluation

Benjamin Feuer, Lucas Rosenblatt, Oussama Elachqar

Abstract

As AI models progress beyond simple chatbots into more complex workflows, we draw ever closer to the event horizon beyond which AI systems will be utilized in autonomous, self-maintaining feedback loops. Any autonomous AI system will depend on automated, verifiable rewards and feedback; in settings where ground truth is sparse or non-deterministic, one practical source of such rewards is an LLM-as-a-Judge. Although LLM judges continue to improve, the literature has yet to introduce systems capable of enforcing standards with strong guarantees, particularly when bias vectors are unknown or adversarially discovered. To remedy this issue, we propose average bias-boundedness (A-BB), an algorithmic framework which formally guarantees reductions of harm/impact as a result of any measurable bias in an LLM judge. Evaluating on Arena-Hard-Auto with four LLM judges, we achieve (tau=0.5, delta=0.01) bias-bounded guarantees while retaining 61-99% correlation with original rankings across formatting and schematic bias settings, with most judge-bias combinations exceeding 80%.

Introduction

The "LLM-as-a-Judge" paradigm has become essential for scaling AI evaluation, yet it remains plagued by technical sensitivities. These implicit biases—ranging from response formatting to option ordering—can lead to "reward hacking" in autonomous systems. While previous attempts at debiasing have relied on prompt engineering or fine-tuning, these methods lack formal guarantees. This research introduces a framework to mathematically bound the impact of measurable biases, treating bias as a form of epistemic uncertainty that can be managed through calibrated noise.

The Bias-Bounded Evaluation (BBE) Framework

The core of the proposed approach is Average Bias-Boundedness (A-BB), a framework inspired by differential privacy. Rather than focusing on worst-case global sensitivity, which is often too conservative for LLMs, A-BB utilizes local sensitivity estimation to provide computationally feasible guarantees.

Methodology

  1. Sensitivity Estimation: The framework samples "neighboring" contexts (e.g., the same response with different formatting) to measure the judge's score fluctuations, defined as the RMS sensitivity.
  2. Calibrated Gaussian Noise: Gaussian noise is injected into the final score, calibrated such that the probability of a bias shifting the score beyond a threshold is kept below a user-defined failure rate ($\delta$).
  3. Lipschitz Shrinkage: To maintain signal integrity, a deterministic shrinkage step compresses the score distribution toward a center point, reducing effective sensitivity and allowing for tighter guarantees.
Comparison of A-BB properties against existing evaluation frameworks like ToE.

As shown in the comparison above, A-BB provides unique advantages including guarantees on all evaluations, the ability to handle unknown biases, and direct bounding of bias impact without requiring human labels.

Experimental Results

The framework was evaluated using the Arena-Hard-Auto benchmark, testing four judge models: GPT-4o-mini, QwQ-32B, DeepSeek-R1-Distill-32B, and GPT-3.5-Turbo.

Signal Preservation and Variance Reduction

A critical finding is that debiasing does not necessitate the destruction of the underlying signal. Across various judges and aggregation strategies, the debiased scores maintained high correlation with original rankings. For instance, the QwQ-32B judge showed significant variance reduction, suggesting that extreme scores in original evaluations were often products of bias-induced inflation rather than genuine model performance.

Heatmap showing signal preservation (correlation) across different judges and aggregation strategies.

As illustrated in the heatmap, most judge-bias combinations achieved signal preservation exceeding 80%, with some reaching as high as 99.9%. This demonstrates that the BBE framework can successfully bound bias while keeping the underlying model rankings almost entirely intact.

Discussion and Limitations

While BBE provides a robust mechanism for bounding bias impact, it is important to distinguish between bias and accuracy. A judge may be perfectly unbiased (consistent across formats) yet still fundamentally incorrect in its qualitative assessment. Furthermore, the framework's efficacy is tied to the measurability of bias; it can only bound biases that can be simulated via a "neighbor generator."

Future work will focus on incorporating uncertainty from the sensitivity estimation itself into the failure budget using concentration inequalities, further strengthening the formal guarantees provided by the framework.

Summary updated

Abstract

As AI models progress beyond simple chatbots into more complex workflows, we draw ever closer to the event horizon beyond which AI systems will be utilized in autonomous, self-maintaining feedback loops. Any autonomous AI system will depend on automated, verifiable rewards and feedback; in settings where ground truth is sparse or non-deterministic, one practical source of such rewards is an LLM-as-a-Judge. Although LLM judges continue to improve, the literature has yet to introduce systems capable of enforcing standards with strong guarantees, particularly when bias vectors are unknown or adversarially discovered. To remedy this issue, we propose average bias-boundedness (A-BB), an algorithmic framework which formally guarantees reductions of harm/impact as a result of any measurable bias in an LLM judge. Evaluating on Arena-Hard-Auto with four LLM judges, we achieve (tau=0.5, delta=0.01) bias-bounded guarantees while retaining 61-99% correlation with original rankings across formatting and schematic bias settings, with most judge-bias combinations exceeding 80%. The code to reproduce our findings is available at https://github.com/penfever/bias-bounded-evaluation.
BibTeX
@misc{feuer2026towards,
  author = {Benjamin Feuer and Lucas Rosenblatt and Oussama Elachqar},
  title = {{Towards Provably Unbiased LLM Judges via Bias-Bounded Evaluation}},
  year = {2026},
  eprint = {2603.05485},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2603.05485},
}
Figure