Calibrated Learning to Defer with One-vs-All Classifiers

Rajeev Verma, Eric T. Nalisnick

In high-stakes domains like healthcare and autonomous driving, AI systems should not always have the final say. The Learning to Defer (L2D) framework addresses this by allowing an AI to "pass the baton" to a human expert when it predicts the human is more likely to make the correct decision.

However, for this human-AI collaboration to succeed, the AI must be strictly calibrated. If the system estimates that an expert has a 70% chance of being right, the expert should empirically be right 70% of the time. This paper identifies a fundamental mathematical flaw in current state-of-the-art L2D systems and proposes a robust, calibrated alternative.

The Calibration Crisis in Softmax Models

The authors discovered that the leading multiclass L2D framework (Mozannar & Sontag, 2020) relies on a softmax parameterization that is inherently "degenerate." Because of how the underlying math is structured, the AI’s estimate of expert correctness is unbounded.

In practice, optimization often pushes the rejection probability past valid limits. This results in pathological "probabilities" that frequently exceed 1.0 (or 100%), leading to severe overconfidence, a breakdown in trust, and a "false mode" at zero error that hides potential risks from decision-makers.

Figure 1: The softmax approach produces invalid probabilities (>1.0) and exhibits significant miscalibration (ECE=7.58%).

A One-vs-All (OvA) Solution

To fix this degeneracy, the authors replace the standard softmax-based loss with a One-vs-All (OvA) surrogate loss.

Instead of framing the task as one large competition between all possible classes and the "defer" option, the OvA approach treats the problem as $K+1$ independent binary classification tasks—one for each of the $K$ classes, plus one specifically for the rejection/deferral option.

This approach has two major advantages:

  1. Natural Bounding: Unlike the softmax version, the OvA parameterization naturally restricts the estimated probability of expert correctness to the valid $[0, 1]$ range.
  2. Theoretical Grounding: Using the method of Error Correcting Output Codes (ECOC), the authors prove that this OvA approach is a "consistent surrogate." As the model ingests more data, it is mathematically guaranteed to converge to the optimal decision rule.

Empirical Validation: Better Calibration for Free

A common fear in machine learning is the accuracy-calibration trade-off—the idea that making a model more reliable will inherently make it less accurate. The researchers evaluated their OvA method across four diverse domains: CIFAR-10 (image recognition), HAM10000 (medical skin lesion diagnosis), Galaxy-Zoo (scientific classification), and HateSpeech (text moderation).

The results demonstrate that calibration comes at no cost to accuracy:

  • Superior Calibration: On CIFAR-10 simulations, the OvA method achieved an Expected Calibration Error (ECE) of 3.01%, less than half the 7.58% ECE of the softmax-based state-of-the-art.
  • Competitive Accuracy: Across all datasets, the OvA method consistently matched or exceeded the accuracy of all baselines, including differentiable triage and standard confidence-based rejection.
  • Faithful Risk Distribution: In sensitive tasks like medical diagnosis, the OvA model's predicted error closely matched the true error, providing a much more transparent assessment of risk.
Figure 2: The proposed OvA method significantly improves calibration, reducing the Expected Calibration Error to 3.01%.

Limitations and Future Directions

While the OvA framework provides a significant leap forward for L2D reliability, the authors note that it may face challenges in scenarios with extreme data scarcity or severe class imbalance compared to traditional multiclass approaches. Furthermore, while this study focused on surrogate loss functions, future work should investigate the calibration of other L2D paradigms, such as reinforcement learning-based policy learning methods. Ultimately, ensuring that AI systems know when to step aside is just as critical as ensuring they know what to predict.

Summary updated

Abstract

The learning to defer (L2D) framework has the potential to make AI systems safer. For a given input, the system can defer the decision to a human if the human is more likely than the model to take the correct action. We study the calibration of L2D systems, investigating if the probabilities they output are sound. We find that Mozannar & Sontag’s (2020) multiclass framework is not calibrated with respect to expert correctness. Moreover, it is not even guaranteed to produce valid probabilities due to its parameterization being degenerate for this purpose. We propose an L2D system based on one-vs-all classifiers that is able to produce calibrated probabilities of expert correctness. Furthermore, our loss function is also a consistent surrogate for multiclass L2D, like Mozannar & Sontag’s (2020). Our experiments verify that not only is our system calibrated, but this benefit comes at no cost to accuracy. Our model’s accuracy is always comparable (and often superior) to Mozannar & Sontag’s (2020) model’s in tasks ranging from hate speech detection to galaxy classification to diagnosis of skin lesions.
BibTeX
@inproceedings{verma2022calibrated,
  author = {Rajeev Verma and Eric T. Nalisnick},
  title = {{Calibrated Learning to Defer with One-vs-All Classifiers}},
  booktitle = {International Conference on Machine Learning},
  year = {2022},
  url = {https://openreview.net/forum?id=acOfwl5NIE},
}
Figure