TL;DR
Researchers from Google have identified a critical flaw in current "Learning to Defer" (L2D) systems: existing joint loss functions tend to underfit the training data when there is a fixed cost to calling an expert. To fix this, they propose two post-hoc estimators that decouple the base model training from the deferral decision, resulting in significantly better accuracy-cost tradeoffs on standard image benchmarks.
The Problem: The Underfitting Trap in Joint Training
In many high-stakes scenarios, we don't just want a single AI model; we want a system that knows its limits. The Learning to Defer (L2D) paradigm allows a base classifier to either make a prediction itself or "defer" the task to a more capable (but expensive) expert—like a human doctor or a massive, computationally heavy model.
The challenge is balancing accuracy against the cost of calling that expert. Previous state-of-the-art methods tried to solve this by training the classifier and the "deferral logic" simultaneously using a joint loss function. However, the authors discovered a subtle limitation: when there is a fixed cost ($c_0$) to invoking the expert (e.g., a flat fee or a baseline latency hit), these joint losses implicitly apply a high level of label smoothing.
This mathematical quirk confuses the model during training. As the fixed cost increases, the probability margin shrinks, making it harder for the model to distinguish between classes. Consequently, the model underfits the data and performs poorly even on "easy" samples it should have mastered.
Proposed Methods: Post-hoc Estimators
To resolve the underfitting issue, the authors move away from joint training and propose a two-stage, post-hoc approach. The core idea is to train a high-quality base model first and then layer the deferral logic on top. They propose two specific strategies:
- Post-hoc Threshold Correction: The system first learns to estimate the probability of the true label and the probability of the expert being wrong. It uses a novel hybrid loss called SOvA (Softmax + One-versus-All) to get calibrated estimates. At inference time, it defers only if the expected error of the base model exceeds the expected cost of the expert.
- Post-hoc Rejector Training: Here, the base model's parameters are "frozen" after initial training. A separate binary "rejector" model is then trained specifically to predict whether it is beneficial to invoke the expert based on the base model's internal state (logits).
By separating these tasks, the base model is never "distracted" by the deferral cost during its initial learning phase, successfully avoiding the underfitting trap.
Key Insights and Results
The researchers evaluated their methods on several standard computer vision benchmarks adapted for the L2D setting, including CIFAR-10, CIFAR-100, and ImageNet.
- Overcoming Underfitting: On CIFAR-100, the authors showed that as the fixed cost $c_0$ increases, the training accuracy of traditional joint losses plummets, while their post-hoc methods remain stable.
- Superior Tradeoffs: In both L2D and Adaptive Inference (where the goal is to save computational FLOPs), the post-hoc estimators consistently achieved higher accuracy for the same "budget" compared to previous state-of-the-art methods like CSS and OvA.
- Low-Deferral Regime: The performance gains were particularly pronounced when the cost of the expert was high, meaning the system only defers on the hardest samples. In this regime, the post-hoc methods significantly outperformed the standard "Chow's rule" baseline.
Limitations and Future Work
While highly effective, the two-stage dependency of the post-hoc approach means its success relies heavily on the quality of the base model produced in the first stage. If the base model is fundamentally flawed, the second-stage deferral logic can only do so much to compensate. Additionally, while the post-hoc operations are simple, they do require an additional training pass for the rejector or the SOvA statistics.
Future directions suggested by the authors include exploring other joint surrogates that might avoid underfitting without needing a two-stage process, potentially looking into non-differentiable losses or complex constraints on model logits.
Summary updated