An end-to-end fairness framework based on counterfactual reasoning: auditing, diagnosis, and mitigation
Lists
Only you see this
Abstract
Artificial intelligence regulations typically require that sensitive attributes (such as gender and race) be excluded from algorithmic decision-making to prevent discrimination, a principle commonly referred to as ''fairness through unawareness.'' However, even when sensitive attributes are removed, algorithmic models may still infer such information through proxy variables that are pervasive in data, often via complex nonlinear relationships, thereby perpetuating or even amplifying systemic bias. To address the problem of indirect discrimination under fairness through unawareness, this paper proposes an end-to-end framework that integrates discrimination auditing, diagnosis, and mitigation. First, by incorporating an advanced Transformer-Based Counterfactual Explainer (TABCF), our framework constructs a more reliable bias auditing system capable of accurately uncovering discriminatory behaviors in models. Second, once bias is detected, we introduce an innovative two-stage NOCCOShapley diagnostic method that identifies the key proxy variables responsible for discrimination and reveals how the model actually exploits these variables in practice. Finally, to mitigate the identified bias, we implement an adjustable -PCF post-processing strategy that enables a quantifiable trade-off between predictive utility and counterfactual fairness without retraining the model. Notably, we find that when the trade-off parameter is set to the prior probability distribution of the sensitive attribute in the dataset, the model achieves an optimal balance between fairness and utility. Extensive experiments on four widely used real-world datasets demonstrate that our end-to-end framework not only outperforms existing methods in auditing and diagnosis, but also provides a practical and effective technical pathway for deploying more responsible and fair AI systems in real-world applications.
BibTeX
@inproceedings{xu2026end,
author = {Yi Xu and Xi Zhang},
title = {{An end-to-end fairness framework based on counterfactual reasoning: auditing, diagnosis, and mitigation}},
booktitle = {Advances},
year = {2026},
doi = {10.54254/2977-3903/2026.31838},
}