TL;DR
Large Language Models (LLMs) are notorious for being confidently wrong, making uncertainty quantification crucial for high-stakes deployments. This paper introduces Clustered Self-Assessment, a simple yet highly effective method that groups an LLM's sampled responses into semantic clusters and prompts the model to evaluate them via a structured multiple-choice question. The resulting token probabilities provide highly accurate, human-interpretable confidence scores that outperform existing techniques while requiring as few as two additional samples.
What Problem Does This Paper Address?
While LLMs excel at generating fluent and persuasive text, they frequently produce outputs that are factually incorrect—a phenomenon widely known as hallucination. This problem is exacerbated by the fact that LLMs lack explicit, reliable uncertainty estimates, making it difficult for users to know when to trust a model's output.
Prior attempts to quantify uncertainty generally fall into two categories:
- Direct Verbalization: Asking the model "Are you sure?" or to output a confidence percentage. Due to inherent model biases, LLMs are notoriously overconfident here, frequently claiming 100% certainty on incorrect answers.
- Sampling-Based Discrepancy: Generating multiple answers and measuring their variation (e.g., Semantic Entropy). While more effective, these methods yield indirect scores that are difficult for humans to interpret and require a high number of samples, making them computationally expensive.
The authors bridge this gap by creating a method that leverages the LLM's internal understanding of its own knowledge while outputting a clean, well-calibrated probability score.
Methods
The authors propose a two-stage pipeline to extract reliable confidence metrics:
- Answer Clustering: For a given question, the LLM samples multiple answers using temperature sampling. Because the same answer can be phrased differently (e.g., "Paris", "It's Paris", "France's capital Paris"), an external Natural Language Inference (NLI) model evaluates pairwise semantic relationships to group equivalent answers into distinct clusters.
- Self-Assessment via MCQ: These distinct semantic clusters are transformed into options for a structured multiple-choice question (MCQ). Crucially, the authors add a "None of the above" option to capture instances where all generated options are incorrect or the model is highly uncertain. This MCQ is fed back to the LLM, and the exact token probability assigned by the model to its original answer choice is extracted as the final confidence score.
By merging semantically compatible generations beforehand, the method prevents identical answers from splitting the model's probability mass, which would otherwise degrade the reliability of the uncertainty estimate.
Datasets
The method was evaluated across two distinct tasks using three widely adopted datasets:
- TriviaQA (TQA) and Natural Questions (NQ): Standard short-answer question-answering datasets used to evaluate factual knowledge retrieval.
- Extreme Summarization (XSum): A dataset featuring longer text outputs, used to evaluate the method's performance on longer, free-form generation tasks.
Key Insights and Ideas
- Structuring the Self-Reflection: LLMs are poor open-ended self-evaluators but excellent multiple-choice test-takers. Forcing the model to choose from a deduplicated list of its own generated hypotheses unlocks its latent ability to accurately weigh its internal knowledge states.
- Deduplication is Non-Negotiable: If you present an LLM with "Paris" and "The capital of France" as separate options, it splits its probability mass between them. Clustering via NLI acts as a crucial semantic deduplication step that ensures the final softmax probability reflects true confidence.
- The Safety Valve Option: Including "None of the above" provides a vital out-of-distribution escape hatch for the model, significantly boosting calibration accuracy when the model genuinely doesn't know the answer.
Key Results
State-of-the-Art Performance
Evaluated across seven open-source models (including the Qwen2.5, Qwen3, and Gemma-3 series), Clustered Self-Assessment consistently achieved the highest AUROC (Area Under the ROC Curve) scores compared to standard baselines like Semantic Entropy, P(True), and SAR.
High Sample Efficiency
Traditional sampling-based uncertainty methods typically require up to 16 additional generations to calculate stable entropy scores. In contrast, Clustered Self-Assessment achieves competitive performance with just two additional samples, drastically reducing the computational overhead typically associated with uncertainty quantification.
Superior Calibration
The confidence scores generated by this method map tightly to empirical accuracy. It achieved significantly lower Brier scores (a metric evaluating the accuracy of probabilistic predictions) than standard probability baselines, proving that its confidence metrics are highly reliable and actionable for human users.
Limitations and Future Work
- Logit Access Dependency: The method relies on extracting exact token probabilities from the model's softmax layer. This limits its applicability to open-source models or APIs that explicitly expose logit data, making it difficult to use with completely closed-source "black-box" APIs.
- External NLI Overhead: Relying on a secondary NLI model (like DeBERTa) to cluster answers introduces an extra dependency and computational step. The authors note that a promising future direction is leveraging the LLM's internal representations to perform the semantic clustering natively.
- Lack of Post-Hoc Calibration: The raw softmax probabilities are used directly as confidence scores. Implementing an additional post-hoc calibration step could further refine the alignment between the confidence score and actual correctness.
Summary updated