Large Language Models (LLMs) are notoriously overconfident. They can generate a factually incorrect statement—a "hallucination"—with the same authoritative tone as a verified truth. To make these models reliable, we need a way to measure their internal uncertainty.
Traditional uncertainty metrics in machine learning often rely on token-level probabilities (logits). However, as Sebastian Farquhar, Lorenz Kuhn, and Yarin Gal argue in Detecting hallucinations in large language models using semantic entropy, token-level entropy is a poor proxy for truth in generative models. This is because natural language is redundant: a model might be 100% certain of a concept but appear "uncertain" simply because it is choosing between many synonymous ways to phrase it (e.g., "Yes," "Indeed," or "That is correct").
The emerging solution is Semantic Entropy: measuring uncertainty at the level of meaning rather than wording.
The Core Mechanism: Semantic Clustering
The intuition behind semantic entropy is simple: if you ask a model the same question multiple times and it gives you ten different phrasings that all mean the same thing, the model is certain. If it gives you ten answers that contradict each other, it is hallucinating.
To quantify this, researchers use a three-step pipeline:
- Stochastic Sampling: Generate multiple independent responses to the same prompt by sampling from the model (often using a higher temperature to encourage variety).
- Semantic Grouping: Group these responses into clusters based on whether they logically entail one another. As Sebastian Farquhar describes, this is typically done using a secondary Natural Language Inference (NLI) model. If Answer A implies Answer B and vice versa, they belong to the same semantic cluster.
- Entropy Calculation: Instead of calculating entropy over individual tokens, calculate it over the distribution of these clusters.
If the probability mass is concentrated in a single cluster, the semantic entropy is low, signaling high confidence. If the mass is spread across many conflicting clusters, the entropy is high, signaling a likely hallucination.
Why Meaning Beats Tokens
Standard calibration techniques fail because they treat "The capital of France is Paris" and "Paris is the French capital" as two different outcomes. This "dilution of probability mass" makes the model look less confident than it actually is.
Conversely, Zhen Lin and Jimeng Sun show in Generating with Confidence: Uncertainty Quantification for Black-box Large Language Models that semantic-based methods can even outperform "white-box" methods that have access to the model's internal math. By treating responses as nodes in a semantic graph, they use the dispersion of meanings to predict errors more accurately than raw probability scores ever could.
Applications Across Modalities and Tasks
The utility of semantic entropy extends beyond simple text-based Q&A:
- Multimodal Reliability: In VL-Uncertainty, Ruiyang Zhang and colleagues apply this to Vision-Language Models. By perturbing both the image (e.g., blurring) and the text, they measure if the model's interpretation remains semantically consistent.
- Medical Safety: Jakob Nikolas Kather and Daniel Truhn demonstrate in Hallucination Filtering in Radiology Vision-Language Models that semantic entropy can act as a "safety valve," allowing models to trade off the quantity of answers for much higher diagnostic reliability.
- Self-Correction and Training: Methods like EMPO, developed by Qingyang Zhang, use semantic consistency as a reward signal. By encouraging models to minimize their own semantic entropy during training, they can improve reasoning capabilities without needing any labeled "golden" answers.
Statistical Guarantees
Beyond mere detection, semantic entropy is being integrated into rigorous statistical frameworks. Shuo Li and Insup Lee use semantic clustering within Conformal Prediction frameworks like TRAQ. This allows a system to provide a set of potential answers that is guaranteed to contain the correct meaning with a user-specified probability (e.g., 95%), effectively bringing formal risk control to the "black box" of LLM generation.