Stochastic Output Inconsistency in Large Language Models

31 concepts27 papers2 proposals

When we interact with a traditional software application, we expect determinism: the same input should invariably yield the same output. Large Language Models (LLMs), however, break this fundamental assumption. Because they are probabilistic engines, they do not compute a single "right" answer; instead, they generate text token-by-token by sampling from a shifting probability distribution.

This inherent randomness introduces a pervasive challenge known as stochastic output inconsistency. Even when provided with identical prompts under low temperature settings, an LLM can trace entirely different paths through its token space across separate sessions. While this variability can mimic human creativity in casual conversations, it introduces severe reliability and safety risks when these models are deployed in high-stakes, decision-critical environments.

The Mechanics of Generation Drift

To understand why LLMs are inherently unstable, we must look at how a single token choice alters the trajectory of an entire response. When an LLM processes a prompt, it outputs a probability distribution over its vocabulary for the next token. If the model selects a slightly less probable token due to sampling randomness—or due to subtle hardware-level non-determinism during parallel GPU operations—that token is appended to the prompt.

Because LLMs are autoregressive, this new token changes the context for all subsequent predictions. A tiny divergence at token ten can completely alter the logical trajectory of token one hundred. As noted by the authors of Towards large language model-based personal agents in the enterprise: Current trends and open problems, this variance persists even when models are configured with near-zero temperature settings, presenting a major roadblock for enterprise workflows that require predictable execution.

Where Randomness Meets High-Stakes Reality

The practical consequences of this stochastic behavior span across multiple domains, most notably in healthcare, algorithmic reasoning, and security.

Clinical and Medical Risks

In medicine, diagnostic and dosing consistency is a safety requirement. However, research shows that LLMs frequently exhibit unstable clinical logic. For instance, Aaron Chase et al. demonstrated in Large language models management of complex medication regimens: a case-based evaluation that models can provide entirely different dosing strategies or treatment plans when presented with identical medical scenarios.

This lack of reproducibility is documented across various medical tasks:

Algorithmic and Security Fragility

Stochasticity also undermines tasks requiring mathematical or structural precision. Giorgio Severi et al. evaluated this in Ask and You Shall Receive (a Graph Drawing): Testing ChatGPT's Potential to Apply Graph Layout Algorithms, finding that the probabilistic nature of LLMs introduces errors in algorithmic execution because the same prompt yields different structural results across iterations.

In security frameworks, this variance cuts both ways:

Quantifying the Instability

Standard evaluation metrics often obscure this problem by measuring "peak" performance—evaluating whether a model can find the right answer on a single run. However, a model that succeeds on the first run but fails on the next nine is highly unstable in production.

To address this gap, Junnan Liu et al. introduced the G-Pass@k metric in Are Your LLMs Capable of Stable Reasoning?. Instead of treating accuracy as a binary pass/fail metric, G-Pass@k quantifies how consistently a model reproduces correct reasoning chains across $k$ independent generations. Their evaluation revealed that many models with high baseline accuracy suffer from severe reasoning instability, forcing users to rely on unpredictable manual regenerations to achieve a correct output.

Harnessing Stochasticity via Consensus Aggregation

If stochasticity is an inescapable property of autoregressive sampling, how can we build reliable systems? A prominent direction in recent research is to stop fighting the randomness and instead use it as a statistical tool. By sampling the model multiple times, we can treat the outputs as a distribution and aggregate them to find a stable consensus.

The Centroid Approach

For structured or decision-making tasks, we can find the statistical "average" of multiple runs. Sana Ebrahimi et al. proposed this strategy with REQUAL-LM in REQUAL-LM: Reliability and Equity through Aggregation in Large Language Models. By treating the black-box LLM as a stochastic sampler, they generate a pool of candidate responses via Monte Carlo sampling and aggregate them into a single "equitable centroid." This process effectively filters out the probabilistic noise of individual runs, yielding an output that reflects the model's stable, core behavior.

Prompt ───► [ LLM Sampler ] ───► Response 1 ┐
                             ───► Response 2 ┼──► [ Centroid Aggregation ] ──► Stable Output
                             ───► Response 3 │

Fine-Grained Synthesis

While voting methods work well for closed-ended tasks (like multiple-choice or math problems), they fail for free-form text generation like summarization or code generation, where two correct answers might not share the same string representation.

To solve this, Xinglin Wang et al. introduced Fine-Grained Self-Consistency (FSC) in Integrate the Essence and Eliminate the Dross: Fine-Grained Self-Consistency for Free-Form Language Generation. Instead of forcing a vote on the entire text string, FSC breaks down multiple sampled responses into smaller semantic segments. It evaluates the consensus of these individual segments across the generation pool, extracts the most stable and verified components, and synthesizes them into a single, optimized response.

By shifting from single-generation inference to multi-sample consensus, these frameworks transform the inherent weakness of model stochasticity into a mechanism for verification and stability.

History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_model_stochasticity_bf38f117,
  title = {model stochasticity},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/model-stochasticity/rdr_bf38f117a4c848fab55438f578b7eb94},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_model_stochasticity_bf38f117_v1,
  title = {model stochasticity},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/model-stochasticity/rdr_bf38f117a4c848fab55438f578b7eb94/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake