In traditional supervised learning, we assume that for every input, a "ground truth" label eventually exists. We train a model, it makes a prediction, and we compare that prediction to the truth to improve. However, in high-stakes domains like lending, hiring, and medicine, this feedback loop is fundamentally broken.
If a bank approves a loan, they eventually learn if the borrower repaid it. But if the bank rejects the loan, they never learn what would have happened. This is the Selective Labels Problem (also known as "one-sided feedback" or "apple tasting"). Because we only observe outcomes for the individuals we "accept," our datasets become increasingly skewed toward our own past decisions, creating a dangerous blind spot for anyone the system has historically excluded.
The Self-Fulfilling Prophecy of Selection Bias
The core challenge, as described by Jon Kleinberg and Sendhil Mullainathan in Discrimination in the Age of Algorithms, is that our training data is not a random sample of the population. It is a sample filtered through a historical decision-maker.
When a model is trained on this filtered data, it learns to replicate the biases of that filter. If a human recruiter historically rejected qualified candidates from a certain neighborhood, the model never sees "success" labels for that neighborhood. It concludes those candidates are high-risk, not because they are, but because it lacks the data to prove otherwise. This creates what Min Wen calls a "downward spiral" in Algorithms for Fairness in Sequential Decision Making, where certain groups are systematically excluded based on a lack of information rather than a lack of merit.
Breaking the Loop with Optimism
How do we learn about the people we are currently rejecting? One intuitive approach is to borrow a concept from reinforcement learning: exploration.
In Strategic Apple Tasting, Keegan Harris and Zhiwei Steven Wu frame this as a regret-minimization problem. To minimize long-term error, a decision-maker must occasionally "taste" the apples they would otherwise discard.
A more sophisticated version of this is Pseudo-Label Optimism (PLOT), introduced by Aldo Pacchiano and Jakob Nicolaus Foerster in Neural Pseudo-Label Optimism for the Bank Loan Problem. Instead of just randomly accepting people, the model is trained to be "optimistic" about rejected points. It temporarily assigns them positive labels during training, which pushes the decision boundary into the "unknown" territory. If the model is consistently wrong about these optimistic guesses, the feedback loop eventually corrects it; if it was right, it has successfully expanded its reach to a new, profitable segment of the population.
Correcting Bias through Weighting and Imputation
When we cannot afford to experiment—perhaps because a "wrong" decision in medicine or bail is too costly—we must rely on statistical corrections of historical data.
- Inverse Probability Weighting (IPW): Grani A. Hanasusanto and Phebe Vayanos demonstrate in Learning Fair Policies for Multi-stage Selection Problems from Observational Data how to use IPW to re-weight the observed outcomes. By giving more weight to "accepted" individuals who look like those who were "rejected," we can approximate the performance of the full population.
- Label Imputation: In Long-term Fairness with Selective Labels, Giovani Valdrighi and Isabel Valera use reinforcement learning to "fill in" the missing labels for rejected candidates. By matching the distribution of these imputed labels to known population statistics, they ensure that fairness constraints are met even when the ground truth is partially hidden.
Leveraging Expert Variation
A clever way to bypass the selective label problem is to look for "natural experiments" in the data. In many systems, multiple decision-makers (like different loan officers or judges) have different thresholds for acceptance.
Jian Chen and colleagues propose in Learning with Selectively Labeled Data from Multiple Decision-makers that we can use the identity of the decision-maker as an Instrumental Variable (IV). If Judge A is lenient and Judge B is strict, the cases where they disagree provide a window into the "rejected" population. By comparing the outcomes of similar cases handled by different experts, we can recover an unbiased estimate of the true risk across the entire distribution.
Auditing for Fairness
The selective labels problem makes it nearly impossible to use standard fairness metrics like "Equal Opportunity," which requires knowing the True Positive Rate for all groups. If you don't know who would have succeeded among the rejected, you can't calculate that rate.
To solve this, Yahav Bechavod and Aaron Roth suggest using human auditors in Individually Fair Learning with One-Sided Feedback. Instead of relying on missing labels, they use a panel of auditors to flag when two similar individuals are treated differently. This shifts the focus from "did the model get the label right?" to "is the model's decision-making process consistent and fair?"—a crucial distinction when the ground truth is fundamentally unobservable.