Enhancing the Accuracy and Fairness of Human Decision Making

Isabel Valera, Adish Singla, Manuel Gomez Rodriguez

Abstract

Societies often rely on human experts to take a wide variety of decisions affecting their members, from judicial rulings to academic reviews. However, these decisions may be imperfect due to limited experience, implicit biases, or faulty probabilistic reasoning. This paper addresses the problem of improving the accuracy and fairness of the overall decision-making process by optimizing the assignment between experts and decisions. We frame this as a sequential decision-making problem that reduces to a sequence of constrained weighted bipartite matchings. Our algorithms utilize posterior sampling to actively trade off exploitation—selecting assignments that lead to accurate and fair decisions—and exploration—learning about experts' preferences and biases. We demonstrate that these algorithms significantly improve both accuracy and fairness on synthetic and real-world data.

Introduction

Human experts are the gatekeepers of many life-altering decisions. Typically, tasks are assigned to these experts randomly or based on simple availability. This approach fails to account for the inherent fallibility of human judgment, including implicit biases and inconsistent reasoning. Rather than proposing to replace humans with algorithms, this work introduces a machine learning framework to optimize who makes which decision. By intelligently routing cases to the most appropriate experts, we can mitigate systemic bias and improve overall utility.

Modeling Human Decision-Makers

We model experts as threshold-based decision-makers. An expert $v$ observes the probability of a positive outcome and makes a binary decision based on a personal threshold $\theta$. Bias is formally defined as an expert having different thresholds for different demographic groups. For example, a judge might be more lenient toward one race than another, requiring a lower probability of non-recidivism for release.

Sequential Matching Algorithms

The assignment problem is framed as a sequential bipartite matching task. We address two scenarios:

  1. Known Thresholds: If expert biases are known, the problem is a maximum weighted bipartite matching. With fairness constraints (e.g., avoiding Disparate Impact), it becomes a "bounded color matching" problem, solvable with a 1/2 approximation guarantee.
  2. Unknown Thresholds: Since internal biases are usually hidden, we use posterior sampling. By maintaining a Beta distribution for each expert's threshold and updating it with each decision, the system balances learning (exploration) with performance (exploitation).

Experimental Results

The effectiveness of these algorithms was tested on synthetic data and the COMPAS recidivism dataset. In the COMPAS experiments, fictitious judges were created to simulate a realistic pool with varying harshness and racial biases.

Performance comparison of matching algorithms across utility, fairness (disparate impact), and regret metrics.

As illustrated in the experimental results, the proposed matching algorithms consistently outperform random assignment. The "Unknown" algorithm effectively learns expert biases, with its performance converging toward the "Known" threshold baseline over time, achieving sublinear regret ($O(\sqrt{T})$).

Key Insights

  • Diversity as a Resource: A diverse pool of experts is a strength, not a weakness. The algorithm routes specific cases to experts whose natural thresholds align with fair and accurate outcomes.
  • The Failure of Point Estimates: Attempting to estimate thresholds using single point values (like the mean) results in linear regret, meaning the system never reaches optimal performance.
  • Robustness: The system maintains its advantages even when a significant portion (up to 50%) of the expert pool is biased, proving its resilience in flawed real-world environments.

Conclusion and Future Work

This research provides a pragmatic framework for reducing systemic bias by assisting human systems rather than replacing them. Future work will address current limitations, such as the assumption of uniform expert ability, static biases, and the requirement for ground-truth probability models. Expanding the model to include collective group decisions also remains a key area for exploration.

Summary updated

Abstract

Societies often rely on human experts to take a wide variety of decisions affecting their members, from jail-or-release decisions taken by judges and stop-and-frisk decisions taken by police officers to accept-or-reject decisions taken by academics. In this context, each decision is taken by an expert who is typically chosen uniformly at random from a pool of experts. However, these decisions may be imperfect due to limited experience, implicit biases, or faulty probabilistic reasoning. Can we improve the accuracy and fairness of the overall decision making process by optimizing the assignment between experts and decisions? In this paper, we address the above problem from the perspective of sequential decision making and show that, for different fairness notions from the literature, it reduces to a sequence of (constrained) weighted bipartite matchings, which can be solved efficiently using algorithms with approximation guarantees. Moreover, these algorithms also benefit from posterior sampling to actively trade off exploitation---selecting expert assignments which lead to accurate and fair decisions---and exploration---selecting expert assignments to learn about the experts' preferences and biases. We demonstrate the effectiveness of our algorithms on both synthetic and real-world data and show that they can significantly improve both the accuracy and fairness of the decisions taken by pools of experts.
BibTeX
@inproceedings{valera2018enhancing,
  author = {Isabel Valera and Adish Singla and Manuel Gomez Rodriguez},
  title = {{Enhancing the Accuracy and Fairness of Human Decision Making}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2018},
  url = {https://openreview.net/forum?id=BQ6MCi8U9-},
}
Figure