In high-stakes applications—such as clinical diagnosis, aviation, or financial forecasting—a wrong prediction can be far more costly than no prediction at all. Traditional machine learning models are forced to make a choice for every input, often leading to overconfident failures on ambiguous or out-of-distribution data.
To solve this, the paradigm of selective prediction (or selective classification) introduces an option to abstain. By equipping models with a mechanism to say "I don't know," these systems can dynamically trade off coverage (the percentage of cases they choose to answer) against risk (the error rate on those answered cases).
The Risk-Coverage Trade-Off
The mathematical foundation of selective prediction decomposes a system into two distinct components: a standard predictor $f(x)$ and a binary selection function $g(x) \in \{0, 1\}$. If $g(x) = 1$, the system outputs the prediction $f(x)$; if $g(x) = 0$, the system abstains from predicting, potentially deferring the instance to a human expert.
As formalized by Yonatan Geifman et al. in Selective Classification for Deep Neural Networks, deep neural networks can achieve near-perfect accuracy if they are permitted to reject a minority of highly uncertain inputs. This controllable trade-off allows practitioners to define a maximum acceptable risk threshold, automatically filtering out low-confidence inputs before they result in critical errors.
[ Input x ]
│
┌───────┴───────┐
▼ ▼
Predictor f(x) Selection Function g(x)
│ │
│ ├───────► g(x) = 0 ──► [ Abstain / Defer to Human ]
▼ │
[ Candidate ] ◄─────┴───────► g(x) = 1 ──► [ Output Prediction f(x) ]
The practical necessity of this architecture spans multiple domains:
- Aviation: Alexandre Arnold demonstrates that automated systems processing critical aeronautical messages require formal mechanisms to abstain when encountering ambiguous or out-of-distribution text to prevent error propagation (Knowledge extraction from aeronautical messages (NOTAMs) with self-supervised language models for aircraft pilots).
- Healthcare: Joseph Paul Cohen et al. emphasize that incorporating rejection mechanisms based on uncertainty scores is essential to prevent silent failures when deploying machine learning models in clinical settings (Problèmes associés au déploiement des modèles fondés sur l’apprentissage machine en santé).
- Expert Workflows: Allen Nie et al. show that integrating an abstention mechanism into clinical note taggers creates a hybrid workflow, focusing manual human review precisely on the most ambiguous data points (DeepTag: inferring diagnoses from veterinary clinical notes).
Quantifying Uncertainty: How Systems Know to Abstain
The core challenge of selective prediction is designing a reliable selection function $g(x)$. The system must extract a signal that correlates strongly with the likelihood of an error. Researchers have converged on two primary strategies: internal model signals and multi-sample consistency.
Internal Model Signals
The most direct way to evaluate certainty is to probe the model’s internal states during inference. For classification and structured prediction tasks, the entropy of the output distribution serves as a common proxy for ambiguity. For example, Gyubok Lee demonstrates that text-to-SQL models can reliably identify unanswerable questions by applying percentile-based entropy thresholds to the model's output, preventing the generation of hallucinated queries when schema information is missing (EHRSQL: A Practical Text-to-SQL Benchmark for Electronic Health Records).
In multimodal settings, internal signals can be captured using auxiliary probes. Dhananjay Ashok et al. utilize internal activation probes to determine when a vision-language model is likely to hallucinate due to a failure in linking visual features to symbolic knowledge, allowing the system to trigger a warning or abstain before generating text (Can VLMs Recall Factual Associations From Visual References?).
Multi-Sample Consistency
When internal token probabilities are unavailable or uncalibrated, systems can estimate uncertainty by sampling multiple outputs for a single input and measuring their variance.
In computer vision and medical imaging, Adrian Tousignant et al. leverage Monte Carlo dropout to perform stochastic sampling, using the variance across runs to filter out low-confidence clinical predictions (Prediction of Disease Progression in Multiple Sclerosis Patients using Deep Learning Analysis of MRI Data.
For black-box Large Language Models (LLMs), Zhen Lin et al. show that measuring the semantic similarity and graph-based dispersion of multiple generated responses provides a robust response-specific confidence score (Generating with Confidence: Uncertainty Quantification for Black-box Large Language Models). This approach isolates the reliability of a specific generated answer from the general ambiguity of the prompt. Similarly, in visual question answering, the authors of Selectively Answering Visual Questions introduce metrics like AVG BLEU to quantify the semantic consistency across sampled answers, establishing a threshold below which the model abstains to protect user safety.
Calibration and Statistical Guarantees
Raw confidence scores or token probabilities are rarely calibrated out of the box; a model might express 99% confidence on an incorrect prediction. To build trustworthy selection functions, researchers apply calibration frameworks and statistical quality control filters.
Conformal Prediction
Conformal prediction provides a mathematically rigorous way to bound error rates. Instead of a single prediction, it outputs a prediction set that is guaranteed to contain the true answer with a user-defined probability (e.g., $1 - \alpha$).
Supreeth P. Shashikumar et al. apply this to early sepsis prediction in the COMPOSER framework, labeling outlier or out-of-distribution cases as "indeterminate" rather than forcing a high-risk guess, which reduces false alarms by up to 85% (Artificial intelligence sepsis prediction algorithm learns to say "I don't know"). For generative text tasks, Jinhao Duan et al. introduce ConU, which uses conformal prediction to construct statistically validated prediction sets and restricts the final output to the most frequent response within that set, effectively filtering out hallucinations without requiring model fine-tuning (ConU: Conformal Uncertainty in Large Language Models with Correctness Coverage Guarantees).
Learned Calibrators
Rather than relying on static thresholds, secondary models can be trained specifically to manage the decision to abstain. Neeraj Varshney et al. demonstrate that combining internal model confidence with an instance's structural difficulty score to train a secondary calibrator significantly reduces the risk-coverage trade-off, particularly in out-of-domain scenarios where standard probability scores degrade (Towards Improving Selective Prediction Ability of NLP Systems).
In multi-component pipelines, such as open-domain question answering, Shehzaad Dhuliawala shows that jointly calibrating the retriever and reader components using a differentiable Gumbel top-K sampling routine allows retrieval confidence to serve as a critical safety signal, allowing the system to better identify unanswerable questions (Calibration of Machine Reading Systems at Scale).
Abstention and Refusal in Generative Language Models
In generative language models, selective prediction manifests differently than in classification. Instead of suppressing an output entirely, generative models must learn to emit explicit refusal strings, null tokens, or generic placeholders.
Hybrid Selection Scores
To optimize this behavior, Jiefeng Chen et al. developed the ASPIRE framework, which uses parameter-efficient soft prompt tuning to learn task-specific self-evaluation (Adaptation with Self-Evaluation to Improve Selective Prediction in LLMs). ASPIRE computes a hybrid selection score that balances the model's internal token-level generation probability with a global assessment of the answer's correctness:
$$\text{Score} = P_{\text{generation}}(\text{Answer} \mid \text{Query}) \times P_{\text{evaluation}}(\text{Correct} \mid \text{Query}, \text{Answer})$$
Similarly, Farima Fatahi Bayat et al. introduce LITO, an adaptive framework that learns the optimal truth-steering intervention intensity for a given query and triggers a standard refusal string if a classifier predicts that all generated candidates are likely to be incorrect (Enhanced Language Model Truthfulness with Learnable Intervention and Uncertainty Expression).
Test-Time Scaling and Verification
Modern architectures also leverage test-time verification to determine when to abstain. Yinya Huang et al. present Mirror-Critique, which trains a verifier via reinforcement learning to generate detailed text critiques of candidate answers (Critique to Verify: Accurate and Honest Test-Time Scaling with RL-Trained Verifiers). If the aggregated verification score of the top candidate falls below a safety threshold, the system declines to answer, significantly reducing overconfident errors compared to standard majority voting.
[ Query ] ──► [ Generator ] ──► [ Candidate Answers ]
│
▼
[ Mirror-Critique ]
│
▼
[ Verification Score ]
│
┌───────────────┴───────────────┐
▼ ▼
Score < Threshold Score ≥ Threshold
│ │
▼ ▼
[ Abstain ] [ Output Answer ]
In long-document processing, Jan Buchmann shows that the quality of extracted source citations serves as a reliable proxy for response accuracy, enabling selective prediction by abstaining whenever the supporting evidence score is insufficient (Attribute or Abstain: Large Language Models as Long Document Assistants).
Safe Fallbacks
When a model decides to abstain, the implementation of the fallback mechanism determines its utility in production environments:
- Null Tokens: In retrieval-augmented generation (RAG), Min-Sang Kim et al. instruct models to output a specific null token if they lack high confidence in their internal knowledge, preventing the introduction of false information during context generation (QPaug: Question and Passage Augmentation for Open-Domain Question Answering of LLMs).
- Placeholders: For production code and command-line generation, Roshanak Zilouchian Moghaddam et al. demonstrate that defaulting to a generic placeholder when neural confidence falls below a threshold maintains user trust far more effectively than providing incorrect or nonsensical values (Generating Examples from CLI Usage: Can Transformers Help?).
- Ensembles: Frameworks like MoRE, developed by Chenglei Si et al., leverage inter-expert agreement across specialized reasoning models to determine whether to output a consensus answer or withhold the response entirely (Getting MoRE out of Mixture of Language Model Reasoning Experts).
Human Perception and System Reliability
The ultimate goal of selective prediction is to improve human-AI collaboration. How users perceive an abstaining system is critical to its adoption.
Andrea Papenmeier et al. investigated user perceptions of abstaining classifiers, finding that explicit uncertainty signaling (such as a clear "don't know" message) serves as an effective safeguard for human decision quality (Know What Not To Know: Users' Perception of Abstaining Classifiers). By preventing the system from forcing a choice on ambiguous samples, it mitigates automation bias—the tendency for humans to blindly trust automated suggestions—and prompts users to rely on their own judgment, all without damaging the user's trust or the perceived competence of the AI.