Where Is Uncertainty Observed? A Mechanistic Understanding of Uncertainty Quantification in LLMs

Xiaoyu Liang, Virginia Ceccatelli, Aliai Eusebi, Tobias Braun, Nils Loose, Carlo Malagnino, Lorenzo CavallaroarXiv 2026 · arXiv

TL;DR

This paper introduces a mechanistic framework for Uncertainty Quantification (UQ) in Large Language Models (LLMs) by defining four "observation lenses": structural, representational, distributional, and semantical. By mapping UQ methods to where they read signals in the model's computation, the authors reveal a hierarchy of information loss that dictates which methods are sufficient for specific reliability targets like correctness or grounding.

What Problem Does This Paper Address?

Large Language Models are increasingly used in high-stakes domains, yet they remain prone to hallucinations—generating plausible but factually incorrect or ungrounded content. To use them safely, we need robust Uncertainty Quantification (UQ) to signal when a model's response shouldn't be trusted.

While many UQ methods exist, previous taxonomies organized them by the source of uncertainty (e.g., data vs. model) or the estimation mechanism (e.g., entropy vs. sampling). This paper argues that these taxonomies miss a crucial dimension: where in the model's internal computation the uncertainty is actually observed.

The "where" matters because information is lost as it moves from the model's internal components to the final output. If you only look at the final answer (the semantical lens), you might miss a signal of doubt hidden in the attention patterns (the structural lens).

Figure 1: Illustrative failure modes and uncertainty-guided actions: (a) abstain when uncertain about answer reliability; (b) revise an answer when it may not be supported by the context; (c) request clarification when the query is underspecified; and (d) review reasoning when uncertain about the reasoning process.

Methods

The authors draw on mechanistic interpretability to define four successive observation points, or "lenses," along the model's residual stream:

  1. Structural Lens: Observes individual component updates (e.g., specific attention head writes or FFN activations). This is the most granular view.
  2. Representational Lens: Observes the hidden states (the sum of all updates in the residual stream).
  3. Distributional Lens: Observes the probabilities assigned to vocabulary tokens (logits).
  4. Semantical Lens: Observes the distribution over meanings (e.g., clustering "10 cents" and "$0.10" as the same answer).
Figure 2: Four lenses applied to two queries. For the math query, the sampled answers agree on a wrong answer, so semantic uncertainty is low, but the hidden state lies near the uncertain direction. For the opinion query, the token distribution spreads over many phrasings, but the answers share one stance. A colored frame marks the suited lens. Illustrative values from an experiment on Qwen2.5-0.5B-Instruct (Qwen et al., 2025).

The paper establishes a Lens Hierarchy: every inner lens determines the outer ones, but the mapping is "non-injective"—meaning information is discarded at each step. To validate this, they use Decodability Baselines (DecBase)—supervised probes designed to measure how much "correctness signal" can be extracted from each lens.

Datasets

The authors conducted a massive survey of 159 UQ papers (summarized in the table below) and performed proof-of-concept experiments using:

  • Models: Qwen2.5-7B (Instruct and Base) and Llama-3.1-8B-Instruct.
  • Benchmarks:
    • TriviaQA and PopQA for open-domain short-form question answering.
  • MMLU as a multiple-choice sanity check.
  • Surveyed Benchmarks: The paper catalogs dozens of benchmarks used in the literature, categorized by format (MCQA, SFQA, LFQA, etc.).
Table 2: Comparison of UQ surveys, benchmarks, and position papers

Key Insights and Ideas

  • The Information Loss Principle: Information flows outward from structural $\rightarrow$ representational $\rightarrow$ distributional $\rightarrow$ semantical. An outer lens cannot serve as a surrogate for an inner one if the target (like "grounding") depends on the discarded information.
  • Sufficiency vs. Efficiency: Choosing a UQ method is a trade-off. An inner lens might be more "sufficient" (it has more info), but an outer lens might be more "efficient" (it doesn't require white-box access or expensive probing).
  • Grounding requires "Inner" sight: While correctness can be predicted from any lens, grounding (whether a model actually used the provided context) often requires the structural lens to see which attention heads were active.
  • Surrogate Estimators: The authors find that most representational and structural UQ methods are actually "surrogates"—they are trained to predict outer-lens targets (like correctness) using inner-lens signals.

Key Results

  • Monotonic Signal Decrease: In experiments, the ability to decode correctness (measured by AUROC) consistently decreased as they moved from the structural lens to the semantical lens.
  • The "Unembeddable" Signal: They found that a significant portion of the correctness signal in hidden states lies in "tail" directions that the model's unembedding matrix expresses weakly. This explains why probes on hidden states often outperform simple logit-based entropy.
  • Survey Findings: Out of 159 papers, the vast majority (146) target correctness. Targets like faithfulness (agreement between internal and external signals) and underspecification (detecting ambiguous queries) remain significantly under-explored.

Limitations and Future Work

  • Probe Dependency: The empirical results depend on the specific probe family used (logistic regression/MLP). Different architectures might extract information differently.
  • Agentic UQ: The authors highlight a major open problem: how uncertainty propagates in "agentic" frameworks where multiple LLM calls are chained together.
  • Native Inner-Lens Estimators: Most current inner-lens methods rely on external labels (like ground truth correctness). A promising future direction is developing "native" estimators that read the model's internal state directly without needing labeled data, perhaps through causal mechanistic validation.
  • Beyond Correctness: There is a clear need for benchmarks and methods that target reliability beyond just "is the answer right?", specifically focusing on faithfulness and query ambiguity.

Summary updated

Abstract

Knowing when a large language model's (LLM's) response can be trusted is critical for its safe use. Uncertainty quantification (UQ) can help LLMs recognize if their responses may be unreliable. Existing taxonomies organize UQ methods by uncertainty source or estimation mechanism. They leave unresolved the question of where uncertainty is observed in the model's computation. The answer matters because the observation point determines whether an estimator has access to enough information to recover a given target, and how well it can recover it. Drawing on mechanistic interpretability, we define four mechanistic observation lenses: the structural lens observes component updates, the representational lens observes hidden states, the distributional lens observes token probabilities, and the semantical lens observes distributions over meanings. We relate these lenses through the residual stream and order them by the information they carry, which gives a principle for selecting UQ methods for different targets. We survey existing methods through the four lenses. Furthermore, we distinguish evaluation targets, criteria, and metrics to clarify different UQ evaluation protocols. Finally, we identify open problems and research directions to connect uncertainty estimation to the internal computation and downstream use of LLMs.
BibTeX
@misc{liang2026where,
  author = {Xiaoyu Liang and Virginia Ceccatelli and Aliai Eusebi and Tobias Braun and Nils Loose and Carlo Malagnino and Lorenzo Cavallaro},
  title = {{Where Is Uncertainty Observed? A Mechanistic Understanding of Uncertainty Quantification in LLMs}},
  year = {2026},
  eprint = {2611.15735},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.15735},
}
Figure