Managing Non Determinism in LLM Evaluators

43 concepts37 papers2 proposals

Large Language Models (LLMs) have revolutionized how we evaluate complex, open-ended generative tasks. From grading student essays to judging image captions, frameworks like CLAIR and LLMScore have shown that an LLM prompt can replace complex, hand-engineered metrics while achieving high correlation with human judgment.

However, treating an LLM as a traditional evaluation metric introduces a fundamental flaw: traditional metrics are deterministic mathematical functions, whereas LLMs are probabilistic text generators. When an LLM acts as a judge, its stochastic nature means that evaluating the exact same input multiple times can yield entirely different scores. This phenomenon, known as stochastic evaluation inconsistency, threatens the reliability of automated benchmarking, model alignment, and grading systems.

The Mechanics of Evaluator Instability

Why do LLM judges disagree with themselves? At its core, the problem stems from the decoding process. When an LLM generates an evaluation score, it samples tokens sequentially from a probability distribution. Even when the temperature is set to zero to encourage greedy decoding, subtle variations in context, underlying hardware non-determinism, or complex prompt structures can cause the model to diverge down different reasoning paths.

As Lee et al. demonstrate in Evaluating the Consistency of LLM Evaluators, this instability manifests in two distinct ways:

  • Low Self-Consistency: Assigning different scores to the identical input across multiple independent inference trials.
  • Low Inter-Scale Consistency: Failing to maintain a stable metric when the scoring format changes (e.g., switching from a 1-5 scale to a 1-100 scale).

Furthermore, assigning absolute numerical scores is inherently difficult for LLMs because they lack a stable internal rubric. As noted by Bi et al. in LINKAGE, this lack of a fixed scale causes the model's absolute ratings to fluctuate wildly across different evaluation sessions. In complex domains like video captioning, Chen et al. found that some automated benchmarks achieve consistent scoring in less than half of repeated trials, highlighting how severe this noise can become (VidCapBench).

The Real-World Consequences of Noisy Metrics

The stochasticity of LLM judges is not just a theoretical nuisance; it introduces severe vulnerabilities in practical applications.

Unreliable Leaderboards

In The Good, The Bad, and The Greedy, Song et al. point out that most public leaderboards rely on a single evaluation run. This ignores the wide distribution of potential outcomes, meaning a model's high rank might simply be a lucky draw from a stochastic distribution rather than a reflection of genuine capability.

Exploitation and Prompt-Hacking

When deployed in high-stakes scenarios like education, stochasticity undermines fairness. Chiang et al. studied the real-world deployment of an LLM grader in a large course and observed a "cat-and-mouse" game where students repeatedly resubmitted the exact same assignment to exploit the model's non-determinism until a higher grade was randomly generated (Large Language Model as an Assignment Evaluator).

Unstable Model Alignment

During Reinforcement Learning from Human Feedback (RLHF), generative reward models may sample rubrics dynamically. As argued by Gupta et al. in Unveiling Context-Aware Criteria in Self-Assessing LLMs, this probabilistic bias yields inconsistent criteria across runs, destabilizing the reward signal and leading to reward hacking.

Strategies for Taming Stochastic Variance

To transform LLMs into reliable evaluation tools, researchers have developed several architectural and statistical mitigation strategies.

Statistical Ensembling and Expected Values

The most straightforward way to combat stochastic noise is to move away from single-pass generation. In Evaluating Evaluation Metrics, Xiao et al. propose a framework based on measurement theory, demonstrating that averaging scores over multiple iterations effectively dampens non-deterministic noise.

Taking this a step further, Tong et al. design G-VEval to calculate an explicit "expected value" derived directly from the model's token probability distribution, bypassing the variance of free-form text generation entirely. For long-form texts where context shifts exacerbate instability, Yang et al. show that multi-pass score averaging is essential to achieve stable assessments (What Matters in Evaluating Book-Length Stories?).

Multi-Agent Discussion Frameworks

Rather than relying on a single isolated judge, some approaches introduce collaborative deliberation to filter out individual stochastic errors. In MATEval, Li et al. introduce a multi-agent discussion framework where multiple LLM instances debate the merits of a piece of text. This collaborative process forces the agents to provide transparent rationales, catching subtle hallucinations or logical inconsistencies that a single noisy agent would overlook.

Latent and Reference-Based Calibration

Another paradigm shifts the evaluation from open-ended text generation to structured or internal representations. Girrbach et al. introduce "Latent Judges" in Reference-Free Rating of LLM Responses via Latent Information, a method that extracts deterministic quality scores directly from a model's internal probabilities and hidden activations. By avoiding stochastic decoding entirely, this approach achieves high self-agreement and finer granularity than traditional text-based ratings.

[Input Text] ───► [LLM Forward Pass] ───► [Hidden Activations & Token Probabilities] ───► [Deterministic Score]
                                                     (Bypasses Stochastic Decoding)

Alternatively, frameworks like LINKAGE anchor the model's judgment by forcing it to perform listwise ranking against a set of pre-graded reference answers, providing an explicit external "ruler" to stabilize the evaluation scale.

Key Foundational Work

Evaluating Evaluation Metrics: A Framework for Analyzing NLG Evaluation Metrics using Measurement Theory (2023) — contributed 2 concept elements to this direction.
CLAIR: Evaluating Image Captions with Large Language Models (2023) — contributed 1 concept elements to this direction.
Arithmetic Sampling: Parallel Diverse Decoding for Large Language Models (2023) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_stochastic_evaluation_inconsistency_25ef4850,
  title = {stochastic evaluation inconsistency},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/stochastic-evaluation-inconsistency/rdr_25ef4850b7224ffb87d9444afc394003},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_stochastic_evaluation_inconsistency_25ef4850_v1,
  title = {stochastic evaluation inconsistency},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/stochastic-evaluation-inconsistency/rdr_25ef4850b7224ffb87d9444afc394003/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake