In high-stakes domains like medicine, finance, and autonomous driving, a wrong prediction can be catastrophic. To mitigate this risk, classical machine learning often relies on selective classification—a framework that gives models a "reject option," allowing them to abstain from making a prediction when their confidence falls below a certain threshold.
However, traditional selective classification treats the decision to abstain as a black box. A model might output a generic "I don't know" or silently route a case to a human expert without further context. As Andrea Papenmeier et al. demonstrate in Know What Not To Know: Users' Perception of Abstaining Classifiers, while knowing when to stay silent improves overall human-AI team accuracy, generic refusals lack the diagnostic transparency required for meaningful human oversight. Without clear explanations, users cannot distinguish whether a model rejected an input due to a simple data corruption issue, an ambiguous borderline case, or a complete lack of training data.
To bridge this gap, an emerging research direction focuses on rejection transparency—developing mechanisms that explain not just what a model predicts, but exactly why it chooses to withhold a prediction.
Deconstructing Uncertainty: Why Do Models Reject?
To explain a rejection, a system must first identify the root cause of its uncertainty. In Topics in Selective Classification, Andrea Pugnana argues that explainable rejection criteria must distinguish between systemic model limitations and data-specific anomalies. The literature generally splits this uncertainty into two categories:
- Aleatoric Uncertainty (Ambiguity): The input lies near a decision boundary where classes naturally overlap. The model understands the domain but finds the specific instance inherently ambiguous.
- Epistemic Uncertainty (Novelty): The input is an outlier or belongs to an out-of-distribution (OOD) sample that the model was never trained to recognize.
The intuition behind why this distinction matters is clear in clinical decision support. As Johannes Brinkrolf et al. show in Causes of Rejects in Prototype-based Classification Aleatoric vs. Epistemic Uncertainty, geometric decomposition can analytically separate these causes. A medical model rejecting a patient profile because their symptoms overlap with two known diagnoses (aleatoric) requires a completely different clinical response than a rejection triggered because the patient exhibits a rare, completely unseen condition (epistemic).
Similarly, in audio processing, Ziyang Ma et al. point out in Towards Reliable Large Audio Language Model that a robust system should explicitly justify a refusal by distinguishing between environmental audio interference and a fundamental lack of domain-specific knowledge.
Mechanisms for Rejection Transparency
Researchers have developed several distinct methodologies to extract and communicate the reasons behind a model's non-decision.
Counterfactual and Semifactual Boundaries
One of the most intuitive ways to explain a rejection is to define the boundaries of the model's confidence zone using counterfactuals. If a model rejects an input, a counterfactual explanation shows the minimum changes required for the model to feel certain enough to accept it.
In Model Agnostic Local Explanations of Reject, André Artelt et al. leverage local surrogate models to generate feature importance profiles specifically for the reject option. By doing so, they provide actionable feedback on what features drove the uncertainty. To make this process more robust, Clara Punzi et al. introduced the L2loRe framework in L2loRe: a method for explaining the reject option, which explicitly uses the distance to the nearest counterfactual example as an interpretable metric for the rejection logic itself.
Conversely, semifactual explanations take an "even if" approach. As proposed by André Artelt et al. in "Even if ..." - Diverse Semifactual Explanations of Reject, semifactuals demonstrate how an input can be modified without changing the rejection outcome. This maps out the exact shape of the model's "uncertainty zone," helping users understand which feature fluctuations are irrelevant to the model's stubborn hesitation.
Natural Language Rationales and Knowledge Gaps
For Large Language Models (LLMs), rejection transparency often manifests as natural language justifications rather than geometric distances. Traditional uncertainty-driven methods typically force an LLM into a binary refusal. However, Hyuhng Joon Kim et al. note in When to Speak, When to Abstain: Contrastive Decoding with Abstention that integrating natural language rationales directly into the decoding process allows models to explain their internal state dynamically.
This internal monologue serves two critical purposes:
- Verifiable Safety Logic: In knowledge erasure tasks, Junfeng Liao et al. demonstrate in Explainable LLM Unlearning through Reasoning that forcing a model to generate a reasoning trace before refusing an answer allows developers to verify that the model is withholding information due to intended safety boundaries rather than random algorithmic failure.
- Pinpointing Evidence Gaps: In multi-round clinical settings, the KnowGuard framework developed by Xilin Dang et al. (KnowGuard: Knowledge-Driven Abstention for Multi-Round Clinical Reasoning) cross-references an external medical knowledge graph to generate explicit transparency reports. Instead of simply refusing to diagnose, the system explicitly states which clinical evidence gaps forced the abstention.
Rejection Beyond Uncertainty: Constraints and Fairness
Abstention is not always a passive byproduct of low statistical confidence; it can also be an active enforcement of operational constraints or ethical boundaries.
When automated systems defer cases to human experts, the human requires context to act justly. In A Fair Selective Classifier to Put Humans in the Loop, Andrea Pugnana et al. introduce a framework that audits rejections for bias. They argue that true transparency requires the model to state whether a case was rejected due to standard statistical uncertainty or because processing it would trigger a localized fairness violation.
┌───────────────────────────┐
│ Incoming Sample │
└─────────────┬─────────────┘
│
▼
┌───────────────────────────┐
│ Selective Classifier │
└─────────────┬─────────────┘
│
[ Choose Reject ]
│
▼
┌─────────────────────────────────────────────┐
│ Explainable Rejection Layer │
└──────┬───────────────┬───────────────┬──────┘
│ │ │
▼ ▼ ▼
┌────────────┐ ┌────────────┐ ┌────────────┐
│ Epistemic │ │ Aleatoric │ │ Constraint │
│ Outlier │ │ Ambiguity │ │ Violation │
└────────────┘ └────────────┘ └────────────┘
A parallel problem occurs in multi-objective optimization and recommendation systems, where candidates are filtered out of a final recommended set. As analyzed by Cyrille Kone et al. in Constrained Pareto Set Identification with Bandit Feedback, a transparent rejection mechanism must explicitly categorize excluded choices. It must distinguish whether a candidate was discarded because it was fundamentally infeasible (violating hard safety constraints) or simply dominated (suboptimal compared to a superior alternative).
Why Rejection Transparency Works
Ultimately, explaining why a model abstains transforms the reject option from an opaque safety cutoff into an interactive diagnostic tool. By forcing the system to expose the geometry of its uncertainty boundaries, detail its missing data pieces, or flag its ethical constraints, rejection transparency provides human operators with the exact context needed to debug the system, gather better data, or make informed decisions when the AI steps aside.
Key Foundational Work
Topics in Selective Classification (2023) — contributed 2 concept elements to this direction.
Model Agnostic Local Explanations of Reject (2022) — contributed 2 concept elements to this direction.
Explaining Reject Options of Learning Vector Quantization Classifiers (2022) — contributed 1 concept elements to this direction.