TL;DR
While Large Language Models (LLMs) can use natural language phrases (like "fairly certain" or "probably") to express confidence reliably within a specific domain, this capability breaks down when they face new topics. Out-of-distribution shifts cause the internal meaning and relative ranking of these verbal confidence markers to fluctuate wildly, making them unreliable for general uncertainty estimation.
What Problem Does This Paper Address?
When humans communicate uncertainty, we rarely say, "I am 74% confident in this statement." Instead, we use epistemic markers—phrases like "probably," "I think so," or "undoubtedly." For LLMs to be safely deployed in high-stakes fields like medicine or law, they need to communicate their confidence in these same natural, human-readable ways.
While previous research has focused on whether LLM expressions match human expectations, this paper tackles a more fundamental question: Do LLMs even maintain a stable internal mapping for the words they choose? If a model says "probably" on a math test versus a legal exam, does it actually correspond to the same level of underlying accuracy?
Methods
The authors define marker confidence empirically as the actual observed accuracy of a model's answers when it chooses to use a specific epistemic marker. For example, if a model includes the phrase "fairly certain" in 100 answers, and 65 of those answers are correct, the marker confidence for "fairly certain" is 65%.
To evaluate how stably models use these markers, the authors prompt LLMs to answer questions while explicitly incorporating a single verbal marker. They then extract these markers and evaluate them across seven newly proposed metrics spanning three categories:
- Calibration Error (ECE-based): Measures how well marker confidence aligns with actual accuracy within the same domain (
I-AvgECE) versus across different domains (C-AvgECE), compared against a standard numerical confidence baseline (NumECE). - Dispersion (CV-based): Measures how widely spread out or concentrated the confidence scores are within and across datasets (
I-AvgCV,C-AvgCV). - Correlation: Tracks whether marker confidence moves proportionally with overall dataset accuracy (
MAC) and whether the relative ranking of markers remains consistent (MRC).
Datasets
The framework was evaluated using seven diverse datasets requiring specialized knowledge across multiple domains:
- Factual & Commonsense Knowledge: BoolQ, StrategyQA, CSQA
- Mathematical Reasoning: GSM8K
- Medical Reasoning: MedMCQA
- Legal Reasoning: CaseHOLD
- Mixed Knowledge: MMLU
Key Insights and Ideas
- Good In-Domain, Terrible Out-of-Domain: LLMs are surprisingly well-calibrated when testing on the same data distribution they were evaluated on. However, when moving cross-domain, their verbal confidence calibration degrades significantly, often performing worse than simple numerical confidence prompts.
- Shifting Definitions: The meaning of a word changes depending on the context. An LLM saying "quite certainly" might hit 90% accuracy on a straightforward factual dataset but drop to 50% accuracy on a complex legal or mathematical dataset.
- Ranking Chaos: Models fail to maintain a logical hierarchy of words across domains. "Probably" might indicate higher accuracy than "seems" in one dataset, but flip completely in another.
- The "Uncertainty Compression" Problem: LLM marker confidences are highly concentrated. Models rarely use markers that correspond to very low accuracy (under 10%), showing a severe failure in expressing true, deep uncertainty through words.
- Scale Helps: Larger, more powerful models (like GPT-4o and Qwen2.5-32B) exhibit significantly more stable marker rankings and lower cross-domain variance than smaller models.
Key Results
- Calibration Breakdown: Across seven models, the In-domain Average Expected Calibration Error (
I-AvgECE) was low at 8.17% (beating numerical confidence at 16.60%). However, the Cross-domain ECE (C-AvgECE) spiked to 17.73%, proving poor robustness. - Low Ranking Consistency: The average Marker Ranking Correlation (
MRC) was a meager 21.34%, indicating that the relative order of confidence words is highly unstable across different tasks. - Model Performance Correlation: There is a strong negative correlation (-0.88) between a model's overall accuracy and its cross-domain variation (
C-AvgCV), meaning smarter models use these markers more consistently.
Limitations and Future Work
- Idealized Constraints: The study restricts models to closed-source QA tasks, English-only markers, and forces the output of exactly one marker per response, which strips away the natural complexity of human dialogue.
- Lack of True Linguistic Alignment: The results show that LLMs mimic marker distributions from training data rather than possessing a robust, intrinsic understanding of verbal uncertainty.
- Future Directions: Future work should explore how to align verbal confidence with actual model performance under distribution shifts and extend this evaluation to long-form, multi-turn communications.
Summary updated