Abstract
Machine learning was repeatedly proven to provide predictions with disparate outcomes, in which subgroups of the population (e.g., defined by age, gender, or other sensitive attributes) are systematically disadvantaged. Previous literature has focused on detecting such disparities through statistical procedures for when the sensitive attribute is specified a priori. However, this limits applicability in real-world settings where datasets are high dimensional and, on top of that, sensitive attributes may be unknown. As a remedy, we propose a data-driven framework called Automatic Location of Disparities (ALD) which aims at locating disparities in machine learning. ALD meets several demands from machine learning practice: ALD (1) is applicable to arbitrary machine learning classifiers; (2) operates on different definitions of disparities (e.g., statistical parity or equalized odds); (3) deals with both categorical and continuous predictors; (4) is suitable to handle high-dimensional settings; and (5) even identifies disparities due to intersectionality where disparities arise from complex and multi-way interactions (e.g., age above 60 and female). ALD produces interpretable fairness reports as output. We demonstrate the effectiveness of ALD based on both synthetic and real-world datasets. As a result, ALD helps practitioners and researchers of algorithmic fairness to detect disparities in machine learning algorithms, so that disparate -- or even unfair -- outcomes can be mitigated. Moreover, ALD supports practitioners in conducting algorithmic audits and protecting individuals from discrimination.
Introduction
As machine learning (ML) systems are increasingly deployed in high-stakes domains such as hiring, lending, and criminal justice, the risk of systematic disadvantage to specific population subgroups—known as disparate outcomes—becomes a critical concern. Traditional fairness audits often suffer from a "needle in a haystack" problem: they require auditors to pre-specify sensitive attributes (e.g., "gender" or "race") before the audit begins.
This approach fails to capture complex, intersectional biases (e.g., bias against Black women specifically) or to identify precise thresholds in continuous variables like age. The Automatic Location of Disparities (ALD) framework addresses these gaps by providing a data-driven method to discover hidden biases without prior knowledge of which attributes are problematic. By automating the discovery of these subgroups, ALD enables more comprehensive algorithmic audits and helps ensure compliance with emerging regulations like the EU AI Act.
Methods
ALD employs a robust three-step pipeline to identify and validate disparity-inducing attributes:
- Recursive Partitioning (Discovery): The framework splits the dataset. On the first half, it utilizes a forest of conditional inference trees to search for subgroups. These trees identify the strongest statistical associations between predictors and model outcomes, allowing for the exploration of complex multi-way interactions.
- Statistical Hypothesis Testing (Validation): To prevent findings from being mere statistical artifacts, ALD validates discovered subgroups on the second half of the data. It uses $\chi^2$ tests to measure disparity magnitude (e.g., statistical parity) and applies Benjamini-Hochberg corrections to manage the multiple testing problem.
- Audit Report Generation: The final output is an interpretable report where subgroups are ranked by bias magnitude or statistical significance, allowing auditors to prioritize mitigation efforts.
Results and Discussion
Performance on Synthetic Benchmarks
In controlled experiments using synthetic datasets, ALD demonstrated superior performance compared to existing baselines like Multivariate Subset Scan. Specifically, ALD excelled in locating exact "cut-off" points for continuous variables and identifying "hidden" intersectional biases where disparities only emerge through the combination of multiple traits.
Real-World Application
When applied to the Adult Income dataset (predicting earnings >$50k), ALD identified that gender bias was most pronounced for "married men" (husbands), who were significantly more likely to receive high-earning predictions. On the COMPAS recidivism dataset, it flagged not only racial disparities but also specific error rate spikes in smaller subgroups that might otherwise be overlooked.
Conclusion
ALD represents a significant advancement in algorithmic fairness by moving beyond a priori assumptions. While powerful, the framework is limited to attributes present in the training data and identifies associations rather than causal relationships. Future work will focus on handling missing sensitive attributes and integrating causal inference to better understand the origins of the detected biases.
Summary updated