TL;DR
Evaluating machine learning fairness usually requires access to sensitive attributes (like race or gender), which are frequently missing due to privacy regulations. This paper shows that directly plugging in predictions from off-the-shelf auxiliary models causes fairness measurement errors proportional to model error and true bias. Drawing from noisy label learning, the authors derive a closed-form calibration that recovers true fairness metrics without requiring ground-truth demographic data or strict independence assumptions.
What Problem Does This Paper Address?
To audit a model for group fairness—such as Demographic Parity (DP), Equalized Odds (EOd), or Equalized Opportunity (EOp)—practitioners need protected demographic attributes. In industry, strict privacy laws often prohibit collecting or storing these attributes.
A natural workaround is using off-the-shelf auxiliary classifiers (e.g., predicting race from a user's name). However, auxiliary models are imperfect. Existing approaches to correct for noisy demographic proxies rely on restrictive assumptions: having a subset of verified ground-truth labels, access to auxiliary training data, i.i.d. distributions between datasets, or conditional independence between proxy errors and model predictions. This paper removes these constraints, enabling accurate fairness auditing using only pre-trained, off-the-shelf auxiliary models on unlabeled target data.
Methods
The authors reframe fairness evaluation through the lens of noisy label learning:
-
Closed-Form Relationship: The authors unify DP, EOd, and EOp into a matrix formulation $H$ of conditional probabilities. They prove that the true fairness matrix $H$ and the directly measured noisy matrix $\tilde{H}$ relate via an exact linear equation: $$H[:, k] = (T_k^\top \Lambda_p)^{-1} \Lambda_{\tilde{p}} \tilde{H}[:, k]$$ where $T_k$ is the local noise transition matrix conditioned on prediction $k$, $\Lambda_p$ is the diagonal matrix of clean priors, and $\Lambda_{\tilde{p}}$ is the noisy prior diagonal matrix.
-
Training-Free Parameter Estimation (HOCFair): Rather than training another model (which easily collapses to the identity mapping), the authors adapt HOC (High-Order Consensus). HOC decodes $T_k$ and clean priors $p$ purely from the pattern of agreements and disagreements across three noisy demographic predictions (either from three separate auxiliary models or generated via 2-nearest-neighbor features).
- Global vs. Local Transition Matrices:
- Local ($\hat{T}_k$): Estimates transition matrices per target class $k$. Mathematically unbiased, but higher variance when data slices are small or noisy.
- Global ($\hat{T}$): Uses a single overall transition matrix across all classes. Introduces minor approximation bias, but significantly reduces estimation variance.
Datasets
The authors evaluated their framework on two standard benchmarks across multiple target model architectures:
- COMPAS: Tabular recidivism prediction ($Y \in \{0, 1\}$). Sensitive attribute: race (binary: Black vs. non-Black; and multi-class: Black, White, Others). Auxiliary models: three name-based race classifiers (average accuracy $\approx 68.85\%$). Target models: Decision Trees, Random Forests, Gradient Boosting, SVM, Logistic Regression, and Neural Networks.
- CelebA: Face image classification for smile detection ($Y \in \{0, 1\}$). Sensitive attribute: gender. Target model: ResNet-18 (accuracy $\approx 90.75\%$). Auxiliary model: facial gender classifier ($\approx 92.55\%$ accuracy). To satisfy the 3-model HOC requirement, nearest neighbors were computed using various pre-trained facial embeddings (FaceNet, FaceNet512, OpenFace, ArcFace, Dlib, SFace).
Key Insights and Ideas
- Direct evaluation error scales with bias: The error of uncalibrated fairness measurements is strictly bounded by the product of the auxiliary model's error rates and the model's true fairness disparity ($\text{Err} \propto \delta \cdot (e_1 + e_2)$). Highly biased models or inaccurate proxy models suffer the worst mismeasurement if uncalibrated.
- The "Local vs. Global" tradeoff: When auxiliary proxies are weak (like name-to-race on COMPAS), local estimation violates HOC assumptions, making the Global estimator substantially more stable. When auxiliary proxies are accurate (like image-to-gender on CelebA), Local estimation avoids approximation bias and achieves near-perfect calibration.
- Viability for downstream mitigation: Beyond evaluation, the calibrated metrics can serve as constraints in constrained optimization (e.g., via the method of multipliers), improving fairness without sacrificing classification accuracy.
Key Results
- COMPAS (Noisy proxies, biased models): Directly measured baseline metrics exhibited severe normalized errors between $41\%$ and $60\%$. The proposed Global calibration reduced estimation errors by $39.6\%$ to $88.2\%$ across DP, EOd, and EOp.
- CelebA (Accurate proxy): For EOd, local calibration reduced normalized error down to $0.10\% - 0.52\%$ (compared to $4.11\%$ for raw direct evaluation and $6.46\%$ for soft-label baselines).
- Ablation with synthetic noise: When noise was progressively injected into CelebA proxies (reducing accuracy from $92.6\%$ down to $58.5\%$), raw metric errors escalated to nearly $80\%$, while calibrated metrics reliably maintained normalized errors between $5\%$ and $20\%$.
Limitations and Future Work
- Diminishing returns on clean proxies: When auxiliary models are already highly accurate ($>92\%$) and the target model exhibits very low true disparity, calibration errors from estimating $T$ can marginally exceed the tiny error of direct measurement.
- Simplifying assumptions in proofs: Theoretical guarantees demonstrating that calibrated metrics are strictly superior to raw metrics (Corollary 2) are proved under simplified binary and conditionally independent settings.
- Full mitigation pipeline: The authors demonstrated preliminary fairness mitigation on CelebA with a Vision Transformer (ViT-B/8), but broader algorithmic integration into diverse fairness-aware training workflows remains open for future research.
Summary updated