TL;DR
Instead of training models to predict potentially biased historical labels, this paper proposes ranking individuals based on their distance to a hypothetical "North Star"—the perfect candidate. By enforcing monotonic relationships between legitimate features and outcomes, the method ensures "meritocratic fairness" and remains robust even when training data is heavily contaminated by human prejudice.
What Problem Does This Paper Address?
Most machine learning classifiers are trained to minimize error against historical labels. However, in sensitive domains like hiring or loan approvals, these labels are often "imperfect proxies" created by biased human decision-makers. If a bank historically discriminated against a certain demographic, a standard model trained on that data will simply learn to automate that discrimination.
The authors tackle the "selective labels problem": we often lack ground-truth data (e.g., we don't know if a rejected applicant would have been successful). They argue that instead of trying to predict these flawed labels, we should define a fair decision criterion based on "merit"—legitimate features that we know should lead to a positive outcome (like higher test scores or longer employment history).
Methods
The authors introduce a distance-based ranking algorithm that follows four key steps:
- Monotonic Scaling: Legitimate features (e.g., GRE scores) are scaled between 0 and 1. The direction is fixed so that higher values always represent a more "qualified" state relative to the desired outcome.
- The North Star: They define a (potentially hypothetical) "North Star" observation ($\star$) which represents the maximum possible value for every legitimate feature.
- Weighted Distance with Correlation Penalties: They calculate the distance of an individual to the North Star using a modified taxicab metric:
$$d''(i, \star) = \sum_{\ell=1}^{L} \omega_\ell (1 - \tilde{\rho}_\ell) (1 - z_i^{(\ell)})$$
- $\omega_\ell$ (Importance): Weights learned from historical data (via permutation importance) to capture which features actually mattered in the past.
- $\tilde{\rho}_\ell$ (Fairness Penalty): A penalty based on the maximum Spearman rank correlation between a legitimate feature and any protected attribute (like gender). This prevents the model from "exploiting" features that act as proxies for protected groups.
- Ranking and Classification: Individuals are ranked by their distance. A capacity threshold $\alpha$ (e.g., the top 20% of applicants) is then used to assign final positive or negative labels.
Datasets
- German Credit Dataset: A standard benchmark containing 1,000 observations with 20 features. The authors categorized features into protected (gender, age, etc.) and legitimate (credit history, savings, etc.).
- Synthetic Graduate School Admission Data: A custom dataset (1,000 observations) modeled after real-world GRE score distributions. This allowed the authors to precisely control the level of "label bias" ($\zeta$) to test how the model performs as historical discrimination increases.
Key Insights and Ideas
- Abandoning Accuracy as the Primary Metric: In the presence of biased labels, high accuracy is actually a sign of a biased model. The authors prioritize "meritocratic fairness"—the idea that if person A is strictly better than person B in every legitimate category, person A must be ranked higher.
- Individual Fairness by Design: By using a distance metric, the model inherently satisfies the principle of "Fairness Through Awareness" (FTA): similar individuals in the feature space will receive similar distances to the North Star and thus similar rankings.
- The Proxy Trap: Standard "Fairness Through Unawareness" (just removing the gender column) fails because other features (like "job type") often correlate with gender. This method's correlation penalty ($\tilde{\rho}_\ell$) explicitly de-weights these "proxy" features.
Key Results
- Zero Meritocratic Unfairness: On the German Credit data, the proposed method achieved 0% meritocratic unfairness, whereas a standard Logistic Regression model treated 57.5% of individuals unfairly (ranking less qualified individuals higher than more qualified ones).
- Robustness to Bias: In synthetic tests, as the discrimination level ($\zeta$) in the training labels increased, standard models became increasingly biased. In contrast, the ranking approach maintained a constant, high level of group fairness (demographic parity) and zero meritocratic unfairness, regardless of how "poisoned" the training labels were.
- The "Cost" of Fairness: While the method's accuracy against imperfect labels was lower (56% vs 78.5% on German Credit), the authors argue this "drop" is actually the model successfully ignoring historical bias.
Limitations and Future Work
- Domain Expertise Required: The method relies on knowing which features are "legitimate" and their monotonic relationship with the outcome. In some complex fields, these relationships might be non-linear or unknown.
- Feature Interactions: The current distance metric treats features additively. The authors suggest that more sophisticated ways of accounting for feature interactions (e.g., where a high score in one area compensates for a low score in another) is a promising area for follow-up.
- Categorical Data: The method works best with numerical or ordinal data; handling high-cardinality non-binary categorical features requires further refinement.
Summary updated