When we use a Large Language Model (LLM) like GPT-4, we are interacting with a "black box." We can see the text it produces, but we cannot see its "internal monologue"—the probability distributions (logits) or hidden layer activations that indicate how certain the model is about its own words.
In traditional machine learning, we use these internal signals to measure uncertainty. Without them, we face a dangerous "observability gap": a model might state a hallucination with the same authoritative tone as a verified fact. To bridge this gap, researchers are developing methods to quantify uncertainty using only the model's external behavior.
The Consistency Intuition
The most intuitive way to tell if a model is "guessing" is to see if it can tell the same story twice. If you ask a model a question five times and it gives five different answers, it is likely uncertain.
This is the core idea behind SelfCheckGPT, developed by Potsawee Manakul, Adian Liusie, and Mark J. F. Gales. By sampling multiple stochastic responses to the same prompt, their system identifies factual errors based on how much these samples contradict one another. Similarly, Jiaxin Zhang and colleagues introduced SAC³, which uses "semantic-aware cross-check consistency." Instead of looking for exact word matches, SAC³ checks if the meaning of the answers remains stable across different ways of asking the same question.
Zhen Lin and Jimeng Sun took this further by treating these multiple responses as nodes in a semantic graph. By measuring the "eccentricity" of these nodes—how far one answer drifts from the others—they can mathematically predict when a model is likely to be wrong without ever looking at a logit.
Just Ask for Confidence
If we can't see the model's internal math, why not just ask it how sure it is?
Surprisingly, Eric Mitchell and Chelsea Finn found that while Reinforcement Learning from Human Feedback (RLHF) often ruins a model's internal calibration (making its internal probabilities overconfident), the model's verbalized confidence remains remarkably accurate. Simply asking a model to "state your confidence from 0 to 100" can reduce calibration error by up to 50%.
However, this "self-reflection" isn't always perfect. Makoto Yamada and Yao-Hung Hubert Tsai observed a scaling law in this ability: while small models are hopelessly overconfident, larger models are significantly better at "knowing what they know" when prompted correctly.
Probing and Perturbation
Another strategy is to "stress test" the model's response. If a model's answer changes when you slightly paraphrase the question, it's a sign of "atypicality"—the model is operating in a region of its knowledge space where it is unstable.
Lingyong Yan and Weiwei Sun developed a Self-Detection method that combines this consistency check with a measure of how "unfamiliar" the phrasing of a question is to the model. Similarly, Kyomin Jung and Jahyun Koo found that forcing a model to express uncertainty (e.g., by adding the prefix "I am not sure but...") causes hallucinations to vanish or change, while factual answers remain stable.
Surrogate Models and Statistical Guarantees
When the black box is too opaque, we can use a "spy" to watch it. Percy Liang and Vaishnavi Shrivastava demonstrated that open-source surrogate models (like Llama 2) can be used to estimate the confidence of closed models like GPT-4. Often, the weaker model’s "gut feeling" about a prompt is a better indicator of the stronger model's reliability than the stronger model's own self-reports.
For applications requiring high reliability, researchers are turning to conformal prediction. Inseop Lee introduced TRAQ, a framework that uses multiple stochastic passes to generate a set of potential answers. This set is mathematically guaranteed to contain the correct response with a user-defined level of probability, providing the "error bars" necessary for rigorous science and medicine.
By shifting the focus from internal weights to external behaviors—consistency, verbalization, and stability—this research direction allows us to trust "black box" AI in high-stakes environments where "I think so" isn't a good enough answer.