A Ranking Approach to Fair Classification

Jakob Schöffer, Niklas Kühl, Isabel Valera

Abstract

Algorithmic decision systems are increasingly used in areas such as hiring, school admission, or loan approval. Typically, these systems rely on labeled data for training a classification model. However, in many scenarios, ground-truth labels are unavailable, and instead we have only access to imperfect labels as the result of (potentially biased) human-made decisions. Despite being imperfect, historical decisions often contain some useful information on the unobserved true labels. In this paper, we focus on scenarios where only imperfect labels are available and propose a new fair ranking-based decision system based on monotonic relationships between legitimate features and the outcome. Our approach is both intuitive and easy to implement, and thus particularly suitable for adoption in real-world settings. We introduce a distance-based decision criterion, which incorporates useful information from historical decisions and accounts for unwanted correlation between protected and legitimate features. Through extensive experiments on synthetic and real-world data, we show that our method is fair in the sense that it assigns the desirable outcome to the most qualified individuals and removes the effect of stereotypes in decision-making, thereby outperforming traditional classification algorithms.

Introduction

Most machine learning classifiers are trained to minimize error against historical labels. However, in sensitive domains like hiring or loan approvals, these labels are often "imperfect proxies" created by biased human decision-makers. If a bank historically discriminated against a certain demographic, a standard model trained on that data will simply learn to automate that discrimination. This creates a "selective labels problem" where we lack ground-truth data for rejected applicants. Instead of attempting to predict these flawed labels, this research proposes defining a fair decision criterion based on "merit"—legitimate features that should lead to a positive outcome, such as higher test scores or longer employment history.

Methods

The proposed approach ranks individuals based on their distance to a hypothetical "North Star"—the perfect candidate. The algorithm follows four key steps:

  1. Monotonic Scaling: Legitimate features (e.g., GRE scores) are scaled between 0 and 1. The direction is fixed so that higher values always represent a more "qualified" state.
  2. The North Star: A hypothetical observation ($\star$) is defined, representing the maximum possible value for every legitimate feature.
  3. Weighted Distance with Correlation Penalties: The distance of an individual to the North Star is calculated using a modified taxicab metric: $$d''(i, \star) = \sum_{\ell=1}^{L} \omega_\ell (1 - \tilde{\rho}_\ell) (1 - z_i^{(\ell)})$$
    • $\omega_\ell$ (Importance): Weights learned from historical data to capture which features mattered in the past.
  4. $\tilde{\rho}_\ell$ (Fairness Penalty): A penalty based on the maximum Spearman rank correlation between a legitimate feature and any protected attribute (like gender), preventing the model from exploiting proxies for protected groups.
  5. Ranking and Classification: Individuals are ranked by their distance, and a capacity threshold $\alpha$ is used to assign final labels.
Feature categorization and parameters for the German Credit dataset, including importance weights (omega) and correlation penalties (rho).

Datasets

The method was evaluated on two primary datasets:

  • German Credit Dataset: A standard benchmark containing 1,000 observations. Features were categorized into protected (gender, age) and legitimate (credit history, savings, etc.).
  • Synthetic Graduate School Admission Data: A custom dataset (1,000 observations) modeled after real-world GRE distributions, allowing for precise control over the level of label bias ($\zeta$) to test robustness against discrimination.

Results and Discussion

Zero Meritocratic Unfairness

A primary insight of this work is that in the presence of biased labels, high accuracy is often a sign of a biased model. The authors prioritize "meritocratic fairness"—the principle that if person A is strictly better than person B in every legitimate category, person A must be ranked higher. On the German Credit data, the proposed method achieved 0% meritocratic unfairness ($S$), whereas a standard Logistic Regression model treated 57.5% of individuals unfairly.

Performance comparison on the German Credit dataset, highlighting the elimination of meritocratic unfairness (S) compared to traditional Logistic Regression.

Robustness and the "Cost" of Fairness

In synthetic tests, as discrimination in training labels increased, standard models became increasingly biased. In contrast, the ranking approach maintained a constant level of group fairness and zero meritocratic unfairness. While the method's accuracy against imperfect labels was lower (56% vs 78.5% on German Credit), this drop represents the model successfully ignoring historical bias rather than a failure of the algorithm.

Conclusion

By shifting the focus from label prediction to merit-based ranking, this method provides a robust framework for fair decision-making. Future work will address the requirement for domain expertise in defining monotonic relationships and explore more sophisticated ways to account for non-additive feature interactions and high-cardinality categorical data.

Summary updated

Abstract

Algorithmic decision systems are increasingly used in areas such as hiring, school admission, or loan approval. Typically, these systems rely on labeled data for training a classification model. However, in many scenarios, ground-truth labels are unavailable, and instead we have only access to imperfect labels as the result of (potentially biased) human-made decisions. Despite being imperfect, historical decisions often contain some useful information on the unobserved true labels. In this paper, we focus on scenarios where only imperfect labels are available and propose a new fair ranking-based decision system based on monotonic relationships between legitimate features and the outcome. Our approach is both intuitive and easy to implement, and thus particularly suitable for adoption in real-world settings. More in detail, we introduce a distance-based decision criterion, which incorporates useful information from historical decisions and accounts for unwanted correlation between protected and legitimate features. Through extensive experiments on synthetic and real-world data, we show that our method is fair in the sense that a) it assigns the desirable outcome to the most qualified individuals, and b) it removes the effect of stereotypes in decision-making, thereby outperforming traditional classification algorithms. Additionally, we are able to show theoretically that our method is consistent with a prominent concept of individual fairness which states that "similar individuals should be treated similarly."
BibTeX
@inproceedings{schoffer2021ranking,
  author = {Jakob Schöffer and Niklas Kühl and Isabel Valera},
  title = {{A Ranking Approach to Fair Classification}},
  booktitle = {COMPASS},
  year = {2021},
  url = {https://openreview.net/forum?id=m0U8MaJPcI},
}
Figure