SelfReflect: Can LLMs Communicate Their Internal Answer Distribution?

TL;DR

When LLMs are uncertain, they typically output a single answer with an attached confidence score. This paper introduces SelfReflect, a novel metric to evaluate whether an LLM can instead generate a rich, text-based summary of its entire internal distribution of possible answers. The authors reveal a resounding negative result: modern LLMs are fundamentally incapable of natively verbalizing their true internal uncertainties, though they can summarize them if explicitly fed their own sampled outputs.

The Problem: Hiding the Distribution

Currently, if you ask an LLM a question and it isn't entirely sure, uncertainty quantification methods usually just attach a probability to the most likely response (e.g., "The capital of France is Paris, 75% confidence").

Expand

Source, p. 2

However, this hides a wealth of information. What makes up the other 25%? Does the model think it might be Lyon? Or is it confusing France with another country? Ideally, an LLM would be fully transparent, outputting a "self-reflective" string like: "I'm 75% sure it's Paris, but it could also be Toulouse or Marseille."

The roadblock to developing models that can do this is evaluation. How do you mathematically measure if a single summary string faithfully represents a complex, high-dimensional probability distribution over thousands of possible generated strings? Without a benchmark metric, we cannot train or evaluate LLMs to communicate their true internal beliefs.

Methods: The SelfReflect Metric

To solve the evaluation problem, the authors developed the SelfReflect metric. It is based on the statistical concept of "predictive sufficiency" and uses a masked-token (Cloze) task to compare a summary against the model's true internal distribution.

Expand

Source, p. 5

Here is how it works:

1. Sample the Distribution: Draw $N$ independent answers from the evaluated LLM to represent its true internal distribution.
2. The Masked Task: Take a newly generated answer and mask out a key word.
3. The Judge: Ask a "judge LLM" to predict the masked word's probability distribution over the vocabulary. It does this twice: once using the $N$ sampled answers as context, and once using the candidate summary string as context.
4. The Score: Calculate the 1-Wasserstein distance between the two predicted probability distributions.

If the summary string perfectly captures the LLM's internal beliefs, the judge LLM should make the exact same predictions regardless of whether it looks at the summary or the raw samples.

The authors used this metric to benchmark 20 modern LLMs (including Qwen, Llama, Gemma, and Phi families) using different prompting strategies: Greedy (just answering), Basic (prompting for a summary of possibilities), Chain-of-Thought (CoT), and "Sample & Summarize" (explicitly generating $N$ samples, pasting them into the prompt, and asking the model to summarize them).

Key Insights and Results

The evaluation across several QA and RAG datasets (Natural Questions, TriviaQA, MMLU, HotpotQA) yielded several critical insights:

• SelfReflect is a highly accurate metric: Standard "LLM-as-a-judge" prompts and embedding-distance metrics fail to notice when a summary misses subtle probabilities or leaves out minor details. SelfReflect catches these nuances, correctly discriminating good from bad summaries in 98.77% of cases, and good from "almost-good" summaries in 93.20% of cases. It also aligns closely with human judgments (Krippendorff’s $\alpha$ of 0.690).
• LLMs are blind to their own uncertainty: When prompted to think step-by-step (CoT) or list other possibilities, LLMs hallucinate uncertainties. They list plausible-sounding alternatives that do not actually match their true internal probability distributions. Across all 20 tested models, single-decoding methods actually scored worse on the SelfReflect metric than a baseline of just outputting a single greedy answer.
• Reasoning and Finetuning fall short: Even advanced reasoning models or explicitly finetuning a model (via SFT or DPO) to output self-reflective summaries failed to generalize. The models merely learned to mimic the style of an uncertain answer without actually looking inward.
• The "Sample & Summarize" Workaround: The only reliable way to get an LLM to faithfully report its uncertainty is to sample it 50 times, feed those 50 answers back into its context window, and ask it to summarize them. This proves LLMs have the requisite summarization skills, but lack native introspection skills.

Limitations and Future Work

The authors note a few limitations of SelfReflect. The 1-Wasserstein distance scores are relative, requiring baselines for context. Additionally, the metric measures information faithfulness rather than conciseness; an adversarial summary that simply concatenates all 50 raw samples would score perfectly, even though it is useless to a user. Finally, it measures subjective belief reflection, not objective truth.

By releasing SelfReflect as an open-source tool, the authors aim to spur a new sub-field. The ultimate goal is to develop architectural or training breakthroughs that allow LLMs to natively and cheaply output faithful uncertainty summaries in a single pass, bypassing the expensive "Sample & Summarize" workaround.

@inproceedings{kirchhof2026selfreflect,
  author = {Michael Kirchhof and Luca Füger and Adam Golinski and Eeshan Gunesh Dhekane and Arno Blaas and Seong Joon Oh and Sinead Williamson},
  title = {{SelfReflect: Can LLMs Communicate Their Internal Answer Distribution?}},
  booktitle = {International Conference on Learning Representations},
  year = {2026},
  url = {https://openreview.net/forum?id=hOErnDsehG},
}