Jointly Learning to Predict and Defer to Experts

38 concepts22 papers2 proposals

When deploying machine learning models in high-stakes environments like medicine or criminal justice, we rarely let the AI make decisions in total isolation. Instead, the model operates as part of a team, working alongside human experts.

The traditional approach to building these systems focuses on maximizing the standalone accuracy of the machine learning model. Once the model is trained, a separate threshold is applied to its confidence scores: if the model is uncertain, it abstains and hands the task to a human.

However, as Nushi et al. demonstrate, the most accurate standalone model is often not the best teammate. A model optimized in a vacuum may spend its limited representational capacity learning patterns that human experts already understand perfectly, while remaining blind to the complex, edge-case scenarios where humans actually need assistance. To build truly complementary systems, we must shift from training independent predictors to jointly learning to predict and defer.


The Core Mechanism: Joint Optimization

The fundamental insight of the "Learning to Defer" (L2D) framework, introduced by authors such as Madras et al. and Mozannar et al., is that a predictive model and its routing policy should be trained simultaneously within a single architecture.

Instead of minimizing the model's individual error, the system minimizes a joint loss function that evaluates the performance of the final decision-maker—whether that decision-maker happens to be the AI or the human expert.

Input (x) ---> [ Joint Architecture ]
                     /          \
                    /            \
        Classifier f(x)        Deferral Policy d(x)
                    \            /
                     \          /
             Final Team Decision (AI or Human)

Mathematically, let $f(x)$ be the model's prediction, $d(x) \in \{0, 1\}$ be the deferral decision ($1$ meaning defer to the human, $0$ meaning the AI predicts), and $m$ be the human expert's prediction for a true label $y$. The system optimizes an objective that balances both worlds:

$$\mathcal{L}(f, d) = (1 - d(x)) \cdot \text{loss}(f(x), y) + d(x) \cdot \text{loss}(m, y) + \text{cost}(d(x))$$

Optimizing this discrete choice directly is computationally challenging and NP-hard (Lang et al.). To overcome this, Mozannar et al. developed consistent surrogate losses that allow the entire pipeline to be trained end-to-end using standard gradient descent.


Why It Works: Complementary Specialization

To understand why joint optimization is so powerful, consider a machine learning model with limited representational capacity (such as a restricted neural network or a linear classifier).

If this model is trained in isolation, it must spread its parameters thin to cover the entire input space, achieving a moderate level of accuracy everywhere.

Isolated Training:
[ AI tries to learn everything ] -> Capacity spread thin across easy and hard cases.

Joint Training:
[ AI ignores human strengths ] -> Capacity concentrated entirely on human blind spots.

When we switch to joint training, the dynamics change completely. For regions of the input space where the human expert is highly proficient, the loss term $d(x) \cdot \text{loss}(m, y)$ drops to zero when the system defers. Because the team is already correct, the model experiences no gradient pressure to learn these patterns.

As a result, the model can selectively ignore regions where human performance is high (Wilder et al.). It concentrates its limited parameters exclusively on the human's blind spots. This co-optimization forces a natural division of labor: the AI becomes a hyper-specialized expert in exactly the subsets of data where the human is prone to error (Charusaie et al.).


Modeling the Nuances of Human Behavior

A key reason standard confidence-based rejection fails is that it treats human experts as infallible, independent black boxes. In reality, human behavior is biased, noisy, and statistically dependent on the task context. Modern deferral frameworks explicitly incorporate these human quirks into the training loop:

  • Correlated Errors: Humans and AI models often fail on the same difficult instances. Wei et al. introduce a dependent cross-entropy loss that models the joint probability of success and failure between the two agents, preventing catastrophic scenarios where the system defers a task to a human who is likely to make the exact same mistake as the machine.
  • Behavioral Awareness: Humans do not always act rationally when interacting with AI. Yin argues that optimization must account for human engagement behavior, forcing the AI to prioritize accuracy on tasks where humans exhibit overconfidence or rely on faulty heuristics. Similarly, Mahmood et al. show that an AI with lower individual accuracy can produce superior team outcomes if its predictions are targeted precisely at instances where the human partner lacks confidence.
  • Unobserved Information: In historical data, human experts often base decisions on contextual factors that were never recorded in the database. Yin et al. present a confounding-robust deferral policy that uses causal sensitivity analysis to identify when a human possesses a unique informational advantage, routing tasks to them whenever unobserved context gives them the edge.

Scaling to Multiple Experts

The real world rarely features just one generic human expert. Instead, organizations rely on panels of professionals with diverse specialties, varying levels of experience, and different operational costs.

                  /---> Expert A (Cardiologist)
Input ---> Router ----> Expert B (Radiologist)
                  \---> Predictor (AI Model)

To accommodate this, the joint optimization paradigm has been extended from a binary choice (predict vs. defer) to a multi-expert routing problem. Frameworks developed by Keswani et al., Verma et al., and Mao et al. simultaneously train a classifier alongside a multi-class routing mechanism. The system learns to map each incoming instance to its ideal destination—whether that is the internal AI model or the specific human specialist best equipped to handle that particular case.

By optimizing for the strengths and weaknesses of the entire ecosystem during training, these systems ensure that human expertise is deployed exactly where it adds the most value, maximizing both the accuracy and efficiency of the human-AI team.

Key Foundational Work

Consistent Estimators for Learning to Defer to an Expert (2020) — contributed 2 concept elements to this direction.
Predict Responsibly: Increasing Fairness by Learning to Defer (2018) — contributed 2 concept elements to this direction.
Predict Responsibly: Improving Fairness and Accuracy by Learning to Defer (2018) — contributed 2 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_joint_human_ai_optimization_59d0255d,
  title = {joint human-AI optimization},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/joint-human-ai-optimization/rdr_59d0255d8cfe419d83fc68d1b8c7a77f},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_joint_human_ai_optimization_59d0255d_v1,
  title = {joint human-AI optimization},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/joint-human-ai-optimization/rdr_59d0255d8cfe419d83fc68d1b8c7a77f/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake