Evaluating Fairness Without Sensitive Attributes: A Framework Using Only Auxiliary Models

Zhaowei Zhu, Yuanshun Yao, Jiankai Sun, Yang Liu, Hang Li

TL;DR

Evaluating machine learning fairness usually requires access to sensitive attributes (like race or gender), which are frequently missing due to privacy regulations. This paper shows that directly plugging in predictions from off-the-shelf auxiliary models causes fairness measurement errors proportional to model error and true bias. Drawing from noisy label learning, the authors derive a closed-form calibration that recovers true fairness metrics without requiring ground-truth demographic data or strict independence assumptions.

What Problem Does This Paper Address?

To audit a model for group fairness—such as Demographic Parity (DP), Equalized Odds (EOd), or Equalized Opportunity (EOp)—practitioners need protected demographic attributes. In industry, strict privacy laws often prohibit collecting or storing these attributes.

A natural workaround is using off-the-shelf auxiliary classifiers (e.g., predicting race from a user's name). However, auxiliary models are imperfect. Existing approaches to correct for noisy demographic proxies rely on restrictive assumptions: having a subset of verified ground-truth labels, access to auxiliary training data, i.i.d. distributions between datasets, or conditional independence between proxy errors and model predictions. This paper removes these constraints, enabling accurate fairness auditing using only pre-trained, off-the-shelf auxiliary models on unlabeled target data.

Methods

The authors reframe fairness evaluation through the lens of noisy label learning:

  1. Closed-Form Relationship: The authors unify DP, EOd, and EOp into a matrix formulation $H$ of conditional probabilities. They prove that the true fairness matrix $H$ and the directly measured noisy matrix $\tilde{H}$ relate via an exact linear equation: $$H[:, k] = (T_k^\top \Lambda_p)^{-1} \Lambda_{\tilde{p}} \tilde{H}[:, k]$$ where $T_k$ is the local noise transition matrix conditioned on prediction $k$, $\Lambda_p$ is the diagonal matrix of clean priors, and $\Lambda_{\tilde{p}}$ is the noisy prior diagonal matrix.

  2. Training-Free Parameter Estimation (HOCFair): Rather than training another model (which easily collapses to the identity mapping), the authors adapt HOC (High-Order Consensus). HOC decodes $T_k$ and clean priors $p$ purely from the pattern of agreements and disagreements across three noisy demographic predictions (either from three separate auxiliary models or generated via 2-nearest-neighbor features).

  3. Global vs. Local Transition Matrices:
  4. Local ($\hat{T}_k$): Estimates transition matrices per target class $k$. Mathematically unbiased, but higher variance when data slices are small or noisy.
  5. Global ($\hat{T}$): Uses a single overall transition matrix across all classes. Introduces minor approximation bias, but significantly reduces estimation variance.

Datasets

The authors evaluated their framework on two standard benchmarks across multiple target model architectures:

  • COMPAS: Tabular recidivism prediction ($Y \in \{0, 1\}$). Sensitive attribute: race (binary: Black vs. non-Black; and multi-class: Black, White, Others). Auxiliary models: three name-based race classifiers (average accuracy $\approx 68.85\%$). Target models: Decision Trees, Random Forests, Gradient Boosting, SVM, Logistic Regression, and Neural Networks.
  • CelebA: Face image classification for smile detection ($Y \in \{0, 1\}$). Sensitive attribute: gender. Target model: ResNet-18 (accuracy $\approx 90.75\%$). Auxiliary model: facial gender classifier ($\approx 92.55\%$ accuracy). To satisfy the 3-model HOC requirement, nearest neighbors were computed using various pre-trained facial embeddings (FaceNet, FaceNet512, OpenFace, ArcFace, Dlib, SFace).

Key Insights and Ideas

  • Direct evaluation error scales with bias: The error of uncalibrated fairness measurements is strictly bounded by the product of the auxiliary model's error rates and the model's true fairness disparity ($\text{Err} \propto \delta \cdot (e_1 + e_2)$). Highly biased models or inaccurate proxy models suffer the worst mismeasurement if uncalibrated.
  • The "Local vs. Global" tradeoff: When auxiliary proxies are weak (like name-to-race on COMPAS), local estimation violates HOC assumptions, making the Global estimator substantially more stable. When auxiliary proxies are accurate (like image-to-gender on CelebA), Local estimation avoids approximation bias and achieves near-perfect calibration.
  • Viability for downstream mitigation: Beyond evaluation, the calibrated metrics can serve as constraints in constrained optimization (e.g., via the method of multipliers), improving fairness without sacrificing classification accuracy.

Key Results

  • COMPAS (Noisy proxies, biased models): Directly measured baseline metrics exhibited severe normalized errors between $41\%$ and $60\%$. The proposed Global calibration reduced estimation errors by $39.6\%$ to $88.2\%$ across DP, EOd, and EOp.
  • CelebA (Accurate proxy): For EOd, local calibration reduced normalized error down to $0.10\% - 0.52\%$ (compared to $4.11\%$ for raw direct evaluation and $6.46\%$ for soft-label baselines).
  • Ablation with synthetic noise: When noise was progressively injected into CelebA proxies (reducing accuracy from $92.6\%$ down to $58.5\%$), raw metric errors escalated to nearly $80\%$, while calibrated metrics reliably maintained normalized errors between $5\%$ and $20\%$.

Limitations and Future Work

  • Diminishing returns on clean proxies: When auxiliary models are already highly accurate ($>92\%$) and the target model exhibits very low true disparity, calibration errors from estimating $T$ can marginally exceed the tiny error of direct measurement.
  • Simplifying assumptions in proofs: Theoretical guarantees demonstrating that calibrated metrics are strictly superior to raw metrics (Corollary 2) are proved under simplified binary and conditionally independent settings.
  • Full mitigation pipeline: The authors demonstrated preliminary fairness mitigation on CelebA with a Vision Transformer (ViT-B/8), but broader algorithmic integration into diverse fairness-aware training workflows remains open for future research.

Summary updated

Abstract

Although the volume of literature and public attention on machine learning fairness has been growing significantly in recent years, in practice some tasks as basic as measuring fairness, which is the first step in studying and promoting fairness, can be challenging. This is because the sensitive attributes are often unavailable in a machine learning system due to privacy regulations. The straightforward solution is to use auxiliary models to predict the missing sensitive attributes. However, our theoretical analyses show that the estimation error of the directly measured fairness metrics is proportional to the error rates of auxiliary models' predictions. Existing works that attempt to reduce the estimation error often require strong assumptions, e.g. access to the ground-truth sensitive attributes in a subset of samples, auxiliary models' training data and the target data are i.i.d, or some form of conditional independence. In this paper, we drop those assumptions and propose a framework that uses only off-the-shelf auxiliary models. The main challenge is how to reduce the negative impact of imperfectly predicted sensitive attributes on the fairness metrics without knowing the ground-truth sensitive attribute values. Inspired by the noisy label learning literature, we first derive a closed-form relationship between the directly measured fairness metrics and their corresponding ground-truth metrics. And then we estimate some key statistics (most importantly transition matrix in the noisy label literature), which we use, together with the derived relationship, to calibrate the fairness metrics. Our framework can be applied to all popular group fairness definitions as well as multi-class classifiers and multi-category sensitive attributes. In addition, we theoretically prove the upper bound of the estimation error in our calibrated metrics and show our method can substantially decrease the estimation error especially when auxiliary models are inaccurate or the target model is highly biased. Experiments on COMPAS and CelebA validate our theoretical analyses and show our method can measure fairness significantly more accurately than baselines under favorable circumstances.
BibTeX
@inproceedings{zhu2023evaluating,
  author = {Zhaowei Zhu and Yuanshun Yao and Jiankai Sun and Yang Liu and Hang Li},
  title = {{Evaluating Fairness Without Sensitive Attributes: A Framework Using Only Auxiliary Models}},
  booktitle = {International Conference on Learning Representations},
  year = {2023},
  url = {https://openreview.net/forum?id=OKfmDPNPwYF},
}

Loading related content...

Figure