TL;DR
This paper introduces a mechanistic framework for Uncertainty Quantification (UQ) in Large Language Models (LLMs) by defining four "observation lenses": structural, representational, distributional, and semantical. By mapping UQ methods to where they read signals in the model's computation, the authors reveal a hierarchy of information loss that dictates which methods are sufficient for specific reliability targets like correctness or grounding.
What Problem Does This Paper Address?
Large Language Models are increasingly used in high-stakes domains, yet they remain prone to hallucinations—generating plausible but factually incorrect or ungrounded content. To use them safely, we need robust Uncertainty Quantification (UQ) to signal when a model's response shouldn't be trusted.
While many UQ methods exist, previous taxonomies organized them by the source of uncertainty (e.g., data vs. model) or the estimation mechanism (e.g., entropy vs. sampling). This paper argues that these taxonomies miss a crucial dimension: where in the model's internal computation the uncertainty is actually observed.
The "where" matters because information is lost as it moves from the model's internal components to the final output. If you only look at the final answer (the semantical lens), you might miss a signal of doubt hidden in the attention patterns (the structural lens).
Methods
The authors draw on mechanistic interpretability to define four successive observation points, or "lenses," along the model's residual stream:
- Structural Lens: Observes individual component updates (e.g., specific attention head writes or FFN activations). This is the most granular view.
- Representational Lens: Observes the hidden states (the sum of all updates in the residual stream).
- Distributional Lens: Observes the probabilities assigned to vocabulary tokens (logits).
- Semantical Lens: Observes the distribution over meanings (e.g., clustering "10 cents" and "$0.10" as the same answer).
The paper establishes a Lens Hierarchy: every inner lens determines the outer ones, but the mapping is "non-injective"—meaning information is discarded at each step. To validate this, they use Decodability Baselines (DecBase)—supervised probes designed to measure how much "correctness signal" can be extracted from each lens.
Datasets
The authors conducted a massive survey of 159 UQ papers (summarized in the table below) and performed proof-of-concept experiments using:
- Models: Qwen2.5-7B (Instruct and Base) and Llama-3.1-8B-Instruct.
- Benchmarks:
- TriviaQA and PopQA for open-domain short-form question answering.
- MMLU as a multiple-choice sanity check.
- Surveyed Benchmarks: The paper catalogs dozens of benchmarks used in the literature, categorized by format (MCQA, SFQA, LFQA, etc.).
Key Insights and Ideas
- The Information Loss Principle: Information flows outward from structural $\rightarrow$ representational $\rightarrow$ distributional $\rightarrow$ semantical. An outer lens cannot serve as a surrogate for an inner one if the target (like "grounding") depends on the discarded information.
- Sufficiency vs. Efficiency: Choosing a UQ method is a trade-off. An inner lens might be more "sufficient" (it has more info), but an outer lens might be more "efficient" (it doesn't require white-box access or expensive probing).
- Grounding requires "Inner" sight: While correctness can be predicted from any lens, grounding (whether a model actually used the provided context) often requires the structural lens to see which attention heads were active.
- Surrogate Estimators: The authors find that most representational and structural UQ methods are actually "surrogates"—they are trained to predict outer-lens targets (like correctness) using inner-lens signals.
Key Results
- Monotonic Signal Decrease: In experiments, the ability to decode correctness (measured by AUROC) consistently decreased as they moved from the structural lens to the semantical lens.
- The "Unembeddable" Signal: They found that a significant portion of the correctness signal in hidden states lies in "tail" directions that the model's unembedding matrix expresses weakly. This explains why probes on hidden states often outperform simple logit-based entropy.
- Survey Findings: Out of 159 papers, the vast majority (146) target correctness. Targets like faithfulness (agreement between internal and external signals) and underspecification (detecting ambiguous queries) remain significantly under-explored.
Limitations and Future Work
- Probe Dependency: The empirical results depend on the specific probe family used (logistic regression/MLP). Different architectures might extract information differently.
- Agentic UQ: The authors highlight a major open problem: how uncertainty propagates in "agentic" frameworks where multiple LLM calls are chained together.
- Native Inner-Lens Estimators: Most current inner-lens methods rely on external labels (like ground truth correctness). A promising future direction is developing "native" estimators that read the model's internal state directly without needing labeled data, perhaps through causal mechanistic validation.
- Beyond Correctness: There is a clear need for benchmarks and methods that target reliability beyond just "is the answer right?", specifically focusing on faithfulness and query ambiguity.
Summary updated