Introduction
In the pursuit of algorithmic fairness, a fundamental tension exists between two primary goals: Treatment Parity (ensuring a model is "blind" and does not use protected attributes to make decisions) and Impact Parity (ensuring the model produces similar outcomes, such as hiring rates, across different demographic groups).
To reconcile these competing objectives, many researchers have proposed Disparate Learning Processes (DLPs). These algorithms use sensitive information during training to "de-bias" the model but produce a final classifier that remains blind to the sensitive attribute at prediction time.
This paper challenges the popular notion that machine learning models must be "blind" to protected characteristics at test time to be fair. Through theoretical proofs and empirical experiments, the authors demonstrate that DLPs are mathematically suboptimal and can inadvertently harm the very groups they are intended to protect.
Methodology and Datasets
The authors combine decision theory with empirical experiments to dissect the mechanics of DLPs. They mathematically prove that any DLP will result in lower or equal accuracy compared to a model that explicitly uses protected attributes to set group-specific thresholds.
To validate these theoretical findings, the authors compared a state-of-the-art DLP against a simple "optimal thresholding" strategy across several datasets:
- Synthetic Hiring Data: A toy dataset where "work experience" predicts success, but "hair length" acts as a proxy for gender.
- CS Graduate Admissions: Real data from a large US university (9,000 records) with injected synthetic bias.
- Public Benchmarks: UCI Income (Adult), UCI Marketing, UCI Credit, and IBM Employee/Customer Attrition datasets.
Key Insights: The Hidden Costs of "Blindness"
The paper reveals several critical flaws in the DLP approach:
1. Blindness is a Proxy for Discrimination If non-sensitive features (like zip code or hobbies) are correlated with protected attributes, a "blind" DLP will simply learn to use those proxies. This effectively implements "treatment disparity" in disguise, making the model's biases less transparent rather than eliminating them.
2. The "Rational Ordering" Violation Because DLPs are forced to be blind at test time, they cannot "see" group-specific distributions. This forces the model to distort its internal logic, flipping decisions for individuals near the boundary based on irrelevant features.
As seen in the synthetic hiring simulation above, a DLP might reject a highly qualified woman because she has "male-like" short hair, while hiring a less qualified man because he has "female-like" long hair. This violates the principle of rational ordering.
3. Mathematical Suboptimality By restricting the model from using group-specific distributions at test time, DLPs yield a worse trade-off between accuracy and fairness than if the model simply used a different cutoff threshold for each group.
Empirical Results and Real-World Harm
In almost every empirical test, the optimal thresholding strategy (explicit treatment disparity) achieved higher parity for a given level of accuracy than the DLP. For instance, on the UCI Credit dataset, the DLP method dropped accuracy to 0.74 to reach an 85% parity rule. In contrast, the authors' thresholding strategy reached 100% parity while maintaining the same accuracy.
Furthermore, DLPs can violate the "do no harm" principle. Explicit treatment disparity can be constrained to only benefit the disadvantaged group. DLPs, however, often "help" the disadvantaged group by accidentally hurting its most qualified members.
In the graduate admissions case study, the DLP flipped 43 decisions to "non-admit." Alarmingly, five of these were women who were highly qualified but possessed "male-like" feature profiles, demonstrating the qualitative harm of forcing algorithmic blindness.
Limitations and Future Directions
While the paper argues for the technical superiority of explicit treatment disparity (group-specific thresholds), the authors acknowledge the legal ambiguity of this approach. In many jurisdictions, explicit thresholds may be viewed as "affirmative action," which is heavily restricted.
Additionally, the analysis assumes the underlying labels are a "ground truth" of qualification. If the training data itself is tainted by human prejudice, simply adjusting thresholds may not be sufficient.
Ultimately, the authors suggest that separating estimation (calculating an accurate probability) from decision-making (choosing the threshold) is a more robust, transparent path forward than trying to bake fairness constraints directly into a model's weights.
Summary updated