Introduction
In high-stakes applications such as healthcare, criminal justice, and credit scoring, the cost of a false prediction can be catastrophic. Traditional machine learning models are typically designed to provide a prediction for every input, regardless of the model's internal uncertainty. Selective classification—also known as "learning to defer" or the "reject option"—offers a more robust alternative. By allowing a model to abstain from making a prediction when uncertainty is high, practitioners can trade off the percentage of cases covered for significantly higher reliability on the remaining instances.
This research outlines a comprehensive PhD agenda aimed at evolving selective classification into a tool that is model-agnostic, explainable, fair, and robust to real-world distribution shifts.
The Challenge of Trustworthy AI
While the concept of a reject option dates back to the 1970s, modern implementations face several hurdles. Current state-of-the-art methods are often "black boxes" specifically tailored for Deep Neural Networks (DNNs) and evaluated primarily on computer vision tasks. This leaves a significant gap for practitioners working with tabular or text data, who require tools that are not only accurate but also interpretable and equitable.
Methodological Framework
The research proposes a multi-faceted approach to refining selective classification:
1. Model-Agnostic Heuristics
A primary contribution is the development of a method to "lift" any probabilistic classifier into a selective one. This approach utilizes cross-fitting—a technique designed to mitigate bias—and quantile estimation to construct a selection function. This function determines the optimal threshold for abstention, ensuring the model only predicts when its confidence meets a specific reliability criterion.
2. AUC-based Optimization
In many real-world scenarios, such as fraud detection, datasets are highly imbalanced. In these cases, simple accuracy is a poor metric. This research focuses on optimizing the Area Under the ROC Curve (AUC) within the selective framework. The goal is to ensure that as the model rejects more instances (decreasing coverage), the ranking performance (AUC) on the remaining data improves predictably.
3. Causal Embedding for Robustness
To address distribution shifts—where the environment at deployment differs from the training environment—the research explores embedding causal relationships into the model. Unlike standard correlations, causal links are more likely to remain invariant across different contexts, providing a more stable foundation for the selection function.
Key Insights and Theoretical Contributions
The research highlights several critical considerations for the deployment of selective classifiers:
- The Coverage-Performance Trade-off: The fundamental premise is that 100% coverage is often unnecessary. In industrial or clinical settings, it is frequently preferable to have a human expert intervene on 20% of cases if it guarantees near-perfect accuracy on the remaining 80%.
- Explainable Rejection: Rejection should not be an opaque decision. The research advocates for integrating Explainable AI (XAI) to clarify why a model abstains. Distinguishing between outliers (aleatoric uncertainty) and regions where the model lacks training data (epistemic uncertainty) is vital for user trust.
- The Fairness Trap: A significant finding is that increasing abstention can inadvertently magnify social disparities. If a model is consistently less confident about a specific demographic group, it may reject those individuals more frequently, effectively excluding them from automated benefits or services. Addressing this requires baking fairness constraints directly into the rejection logic.
Results and Future Work
Initial results include the development of a new model-agnostic heuristic (Pugnana and Ruggieri 2023) that outperforms existing state-of-the-art methods on real-world tabular datasets. Furthermore, the research provides a theoretical framework ensuring that as coverage decreases, the AUC on the remaining instances improves, offering a guaranteed trade-off for practitioners.
Future work will focus on:
- Empirical Benchmarking: Creating a comprehensive benchmark for selective classification that includes tabular and text data to ensure reproducibility.
- Transparency and XAI: Building selective functions that provide human-understandable rationales for abstention.
- Fairness Constraints: Incorporating explicit fairness metrics into the cross-fitting approach to prevent biased rejection patterns.
- Causal Robustness: Investigating how causal knowledge can validate the stability of a model's confidence function under environmental changes.
Summary updated