Abstract
While machine learning models are usually assumed to always output a prediction, there also exist extensions in the form of reject options which allow the model to reject inputs where only a prediction with an unacceptably low certainty would be possible. With the ongoing rise of eXplainable AI, a lot of methods for explaining model predictions have been developed. However, understanding why a given input was rejected, instead of being classified by the model, is also of interest. Surprisingly, explanations of rejects have not been considered so far. We propose to use counterfactual explanations for explaining rejects and investigate how to efficiently compute counterfactual explanations of different reject options for an important class of models, namely prototype-based classifiers such as learning vector quantization models.
Introduction
In safety-critical fields like medicine or autonomous driving, an incorrect prediction can have catastrophic consequences. In these domains, "I don't know" is often a more valuable answer than a low-confidence guess. This is the role of reject options: mechanisms that allow a machine learning model to withhold a prediction if the input is too ambiguous (near a decision boundary) or too "weird" (out-of-distribution).
However, a rejection without an explanation is often unhelpful and frustrating for the end-user. If a diagnostic tool rejects a patient's data, the clinician needs to know if the rejection stems from corrupted data, a sensor error, or a borderline clinical case that requires a specialist's review. Despite the rapid growth of eXplainable AI (XAI), the question of why a sample was rejected has been almost entirely overlooked. This research addresses this gap by proposing a framework for explaining rejects in prototype-based models.
Methodology: Explaining the "Why" of Rejection
Learning Vector Quantization (LVQ)
The authors focus on Learning Vector Quantization (LVQ), a class of models that classify data by comparing inputs to "prototypes"—representative points in the feature space. LVQ models are inherently more interpretable than many black-box models because their decisions are based on similarity to these known prototypes.
Reject Strategies
The paper investigates three common strategies used by LVQ models to decide when to reject an input:
- Relative Similarity (RelSim): Rejects if the distance to the closest prototype is too similar to the distance to the second-closest prototype of a different class (indicating high ambiguity).
- Decision Boundary Distance: Rejects if the sample is physically too close to the hyperplane separating two classes.
- Probabilistic Certainty: Rejects if the estimated class probability falls below a predefined threshold.
Counterfactual Explanations
To explain these rejects, the authors utilize Counterfactual Explanations. A counterfactual provides actionable feedback by describing the minimal change required to reach a different outcome. In this context, a counterfactual answer looks like: "If feature X were 0.5 units higher, the model would have accepted this input and classified it as Class A."
Technically, the authors formulate these explanations as optimization problems. They demonstrate that for RelSim and Boundary Distance, the "closest" counterfactual can be found efficiently using Convex Quadratic Programming and Linear Programming. For the more complex Probabilistic Certainty, they propose a convex approximation that guarantees a valid explanation even when the underlying probability function is non-convex.
Experimental Evaluation
The researchers evaluated their algorithms across four diverse datasets:
- Wine: Predicting wine cultivators (13 features).
- Breast Cancer: Classifying samples as benign or malignant (30 features).
- Flip: A medical dataset for predicting liver fibrosis (12 features).
- T21: A large, highly imbalanced dataset (50,000+ samples) for early diagnosis of chromosomal abnormalities in pregnancy.
Key Insights
- Sparsity and Interpretability: By using an $L_1$-norm in their optimization, the authors force the explanations to be "sparse," meaning they change as few features as possible. A simple explanation involving one or two variables is significantly more useful to a human practitioner than one requiring simultaneous tweaks to dozens of variables.
- Rejects vs. Predictions: The paper highlights that explaining a reject is mathematically distinct from explaining a class. The goal is not to move the point to a new class, but to move it into the "certainty zone" of any class.
- Efficiency: Because LVQ models are based on discrete prototypes, the authors can break the global optimization problem into smaller, convex sub-problems (one for each prototype), ensuring they find the globally optimal explanation efficiently.
Results and Discussion
The proposed specialized algorithms consistently outperformed "black-box" solvers and baseline methods (such as searching for the nearest accepted training sample) in terms of sparsity. In "ground truth" tests—where the researchers intentionally perturbed specific features to trigger a reject—the counterfactual algorithms identified and "fixed" those exact features nearly 100% of the time for most datasets.
Furthermore, the convex optimization approach provides a level of reliability that general-purpose black-box tools lack; it is guaranteed to find a valid counterfactual if one exists within the model's constraints.
Limitations and Future Work
While this work provides a robust foundation for explaining rejections, it is currently tailored to LVQ and prototype-based models. Future research is needed to adapt these specific mathematical derivations to Deep Neural Networks or Random Forests. Additionally, while the mathematical "goodness" of the explanations is high, user studies are required to confirm that these counterfactuals are truly intuitive and helpful for practitioners in real-world clinical or industrial settings.
Summary updated