Equalized odds postprocessing under imperfect group information

Pranjal Awasthi, Matthäus Kleindessner, Jamie Morgenstern

Abstract

Most approaches aiming to ensure a model’s fairness with respect to a protected attribute (such as gender or race) assume to know the true value of the attribute for every data point. In this paper, we investigate the robustness of the Equalized Odds (EO) postprocessing method when the protected attribute is noisy or corrupted. We prove that under natural conditions, this method still effectively reduces bias and can even lead to better accuracy-fairness trade-offs than using perfect information.

Introduction

The deployment of machine learning in sensitive domains like criminal justice or hiring has spurred the development of algorithmic fairness techniques. Most of these methods rely on a critical assumption: the availability of perfect, "ground truth" labels for protected attributes. In reality, such data is often messy—self-reported with errors, estimated via noisy proxies, or collected by third parties with inherent biases. This paper addresses a critical question for practitioners: if a fairness intervention is run using corrupted group information, does it actually improve fairness or inadvertently exacerbate bias?

Theoretical Framework and Robustness

The researchers focus on the postprocessing method introduced by Hardt et al. (2016), which modifies a classifier's predictions to satisfy fairness constraints via linear programming. To analyze the impact of noise, they introduce a noise model governed by two key assumptions:

  1. Conditional Independence: The noise in the observed protected attribute ($A_c$) is independent of the model's original predictions ($\tilde{Y}$), given the true labels and true attributes.
  2. Moderate Perturbation: The probability of an attribute being mislabeled is not so high that groups become indistinguishable.

Under these assumptions, the authors prove Theorem 1, guaranteeing that Equalized Odds postprocessing with noisy data results in a classifier that is strictly fairer than the original, uncorrected model. This suggests that the EO framework is fundamentally robust to imperfect group information.

Bias and error rates across various synthetic noise scenarios.

Empirical Validation

The theoretical results were validated using synthetic simulations and three real-world datasets: the Drug Consumption dataset, the Adult dataset, and COMPAS.

The Accuracy-Fairness Trade-off

A counterintuitive finding is that using noisy attributes for fairness correction often yields higher accuracy than using perfect attributes. This is because noisy data effectively "loosens" the fairness constraints, preventing the aggressive corrections that typically lead to significant performance drops. In this sense, postprocessing with noisy data acts as an interpolator between the original biased model and a perfectly fair one.

Real-World Performance: COMPAS

On the COMPAS recidivism dataset, the authors demonstrated that their theoretical upper bounds on bias remain tight even when the protected attribute (race) is flipped for up to 50% of the data points. This empirical evidence confirms that "imperfect" fairness interventions can be more palatable for those concerned about performance drops while still achieving significant bias reduction.

Experimental results on the COMPAS dataset showing bias reduction and error stability under increasing attribute noise.

Conclusion and Limitations

This work provides a "green light" for practitioners to apply fairness corrections even when data quality is imperfect. While the current study is limited to binary classification and assumes non-adversarial noise, it establishes that the Equalized Odds framework is remarkably resilient. Future work may extend these guarantees to multi-group scenarios and adversarial noise settings where group labels might be intentionally manipulated.

Summary updated

Abstract

Most approaches aiming to ensure a model’s fairness with respect to a protected attribute (such as gender or race) assume to know the true value of the attribute for every data point. In this paper...
BibTeX
@inproceedings{awasthi2020equalized,
  author = {Pranjal Awasthi and Matthäus Kleindessner and Jamie Morgenstern},
  title = {{Equalized odds postprocessing under imperfect group information}},
  booktitle = {International Conference on Artificial Intelligence and Statistics},
  year = {2020},
  url = {https://openreview.net/forum?id=V-s7C7stlbJ},
}
Figure