Procedural Fairness in Machine Learning

Ke Tang, Ziming Wang, Changwu Huang, Xin Yao

Introduction

In the field of algorithmic fairness, the prevailing focus has been on distributive fairness—ensuring equitable outcomes across different demographic groups. However, research by Ziming Wang and colleagues argues that this "outcome-only" view is incomplete. A model might achieve fair results through biased reasoning, a phenomenon known as "the right results for the wrong reasons." This paper introduces a formal framework for Procedural Fairness, shifting the focus from what a model decides to how it decides.

Methodology: The $GPF_{FAE}$ Framework

The authors define procedural fairness as the requirement that a model applies the same decision process to similar individuals, regardless of their protected group status. To quantify this, they propose the Group Procedural Fairness via Feature Attribution Explanations ($GPF_{FAE}$) metric.

Overview of the $GPF_{FAE}$ pipeline for measuring procedural fairness.

The $GPF_{FAE}$ pipeline involves several key steps:

  1. Pair Selection: Identifying similar individuals from different groups (e.g., $D_1$ and $D_2$) to ensure a baseline for comparison.
  2. Explanation Generation: Utilizing Feature Attribution Explanations (FAE), such as SHAP, to determine the importance (logic) the model assigns to specific features for these individuals ($E_1$ and $E_2$).
  3. Logic Comparison: Calculating the Maximum Mean Discrepancy (MMD) between these explanation sets ($d_\Phi(E_1, E_2)$). A high discrepancy indicates that the model is using divergent logic for different groups, signaling procedural unfairness.

Mitigation Strategies

To address detected biases, the paper suggests two primary interventions:

  • Retraining (Feature Removal): Identifying "Unfair Features" (UFs) that contribute most to logic disparity and retraining the model without them.
  • Modification (Explanation Loss): Incorporating a penalty term into the model's loss function during training. This "explanation loss" discourages reliance on unfair features, incentivizing the model to adopt more equitable reasoning.

Experimental Evaluation and Key Insights

The framework was tested on a synthetic dataset and eight real-world benchmarks, including COMPAS, Adult, and LSAT.

The COMPAS Paradox

A notable finding involved the COMPAS recidivism dataset. While notorious for distributive bias (unequal outcomes), it exhibited relatively high procedural fairness. This suggests the model's bias stems from applying the same (potentially flawed) logic consistently across groups, rather than using different "rules" for different races. This highlights that "fair features" do not inherently guarantee a fair process, as proxy features can still lead to divergent logic.

Performance and Trade-offs

The study demonstrated that improving procedural fairness often yields a synergy effect, naturally enhancing distributive fairness. Furthermore, the cost to predictive performance was minimal:

  • Retraining: Average accuracy drop of only 0.8%.
  • Modification: Average accuracy drop of 1.8%.

Conclusion and Limitations

The $GPF_{FAE}$ metric provides a robust, automated way to audit the "black box" of machine learning logic. A primary limitation is data sparsity; finding "similar" pairs in high-dimensional spaces can be difficult. The authors suggest using counterfactual data generation to create "synthetic twins" for comparison when real data is thin. Future work aims to integrate these metrics directly into initial training phases to handle multiple sensitive attributes simultaneously.

Summary updated

BibTeX
@article{tang2026procedural,
  author = {Ke Tang and Ziming Wang and Changwu Huang and Xin Yao},
  title = {{Procedural Fairness in Machine Learning}},
  journal = {J. Artif. Intell. Res.},
  year = {2026},
  doi = {10.1613/jair.1.20498},
}
Figure