TL;DR
CAELUM is a framework for building semantically calibrated prediction sets for Large Language Models (LLMs) by separating aleatoric uncertainty (ambiguity) from epistemic uncertainty (knowledge gaps). It introduces "matchability"—the probability that a model can even generate the correct answer—as a fundamental theoretical ceiling for reliability and uses a pre-generation classifier to significantly improve coverage guarantees in open-ended tasks.
What Problem Does This Paper Address?
While LLMs are increasingly used in critical settings, they often struggle to communicate how certain they are about an answer. Conformal Prediction (CP) is a popular statistical framework that provides "guaranteed" reliability: if you ask for 90% coverage, it gives you a prediction set that contains the true answer 90% of the time.
However, applying CP to open-ended text generation faces two major hurdles:
- Infinite Response Space: Unlike simple classification, the space of possible text responses is vast, and "correctness" is semantic rather than an exact string match.
- The Knowledge Gap: Standard CP assumes the true answer is always reachable. For LLMs, the correct answer might not even be in the model's sampled support (i.e., the model simply doesn't know it). If the model cannot generate the right answer, no amount of statistical thresholding can include it in a prediction set.
The authors identify matchability—the fraction of items where the reference answer actually appears in any sampled cluster—as the "hard ceiling" for LLM reliability.
Methods
The authors propose CAELUM (Calibrated Aleatoric-Epistemic Language Uncertainty Modeling), which follows a five-step pipeline:
- Clarification Ensembles: For a given query, the model generates $n$ "clarification" prompts. This helps capture aleatoric uncertainty (e.g., "What is the capital of South Africa?" is ambiguous because it has three valid capitals).
- Sampling & Semantic Clustering: The model samples $m$ responses for each clarification. These are grouped into semantic clusters (e.g., "Pretoria" and "The city of Pretoria" are the same cluster).
- Uncertainty Decomposition: Using kernel-based von Neumann entropy, CAELUM splits total uncertainty into two components:
- Aleatoric ($U_{ale}$): Disagreement between different clarifications.
- Epistemic ($U_{epi}$): Disagreement within a single clarification (a sign the model is guessing).
- Weighted Conformal Prediction: They use a novel weighting function $w_\lambda(x) = 1 + U_{ale}(x) + \lambda U_{epi}(x)$ to scale the conformal threshold. This allows the system to expand prediction sets more aggressively when the model shows signs of a knowledge gap.
- Matchability Stratification: They train a lightweight binary classifier to predict if a question is "matchable" based only on the question's features (like embeddings and length) before generation.
Datasets
The study evaluates 30 combinations of LLMs (ranging from 4B to 32B parameters, including Qwen3, Ministral, and Gemma) across six benchmarks:
- Factual: TriviaQA, OpenNQ, SimpleQA, and PopQA.
- Ambiguity: AmbigQA and AmbigInst (tasks specifically designed with multiple valid interpretations).
Key Insights and Ideas
- The Matchability Ceiling: You cannot statistically guarantee 90% coverage if your model only generates the correct answer 60% of the time. This paper formalizes this "support limitation" as a theoretical bound on conformal prediction.
- Pre-generation Triage: Remarkably, a simple classifier can often predict if an LLM is likely to fail before it even starts generating, just by looking at the question. This allows for safer routing—for example, sending "unmatchable" questions to a human or a retrieval system.
- Epistemic vs. Aleatoric: Disagreement within a single clarification is a much stronger signal for a "knowledge gap" (hallucination) than disagreement across different clarifications, which usually signals legitimate ambiguity.
- Universal Hardness: Matchability is somewhat universal. A classifier trained on one model's failures can predict another model's failures with high accuracy (mean transfer AUC of 0.78), suggesting that some questions are inherently difficult for current LLM architectures.
Key Results
- Improved Reliability: In the "predicted-matchable" stratum, mean coverage reached 0.74, compared to 0.65 overall, significantly closing the gap toward the 0.90 target.
- Hallucination Detection: The epistemic uncertainty signal ($U_{epi}$) outperformed standard total entropy for ranking support failures (hallucinations) by an average AUROC improvement of +0.014.
- Efficiency: CAELUM achieved these coverage gains while maintaining or reducing the average size of the prediction sets compared to vanilla conformal methods.
Limitations and Future Work
- Computational Cost: Generating 50 responses per query (5 clarifications × 10 samples) is expensive and slow for real-time applications.
- Heuristic Matching: The "ground truth" for whether a model's answer matches a reference relies on heuristic tiers (substrings, ROUGE scores, embeddings), which can still produce false positives or negatives.
- Persistent Gaps: Even with CAELUM, some datasets (like PopQA) remain well below nominal coverage targets, indicating deep knowledge gaps that statistical calibration alone cannot fix.
- Future Directions: The authors suggest extending this work to long-form generation, multi-turn dialogues, and multimodal settings where uncertainty might be spread across different media (e.g., text vs. images).
Summary updated