TL;DR
Aequitas is an open-source bias and fairness auditing toolkit designed to seamlessly integrate into the machine learning workflow for both data scientists and policymakers. It operationalizes a wide variety of fairness metrics across multiple population subgroups, helping teams move from abstract ethical principles to concrete, actionable audits before deploying models in high-stakes public policy domains.
What Problem Does This Paper Address?
As machine learning models are increasingly deployed in critical public policy areas—such as criminal justice, healthcare, and public safety—concerns about unintended bias against protected groups (based on race, gender, age, etc.) have grown. While researchers have proposed numerous mathematical definitions of fairness, there is no universal consensus on which metric to use, and few tools exist to easily operationalize them on real-world datasets.
Furthermore, existing academic research often focuses on simplistic binary setups (e.g., male vs. female) using synthetic or toy datasets. This leaves a massive gap between theoretical fairness and practical application in complex, multi-group policy scenarios where resources are constrained and choices have life-altering consequences.
Methods
Aequitas provides a structured framework to evaluate both unsupervised (distributional) and supervised (error-based) group metrics:
- Distributional Metrics: Evaluates how interventions are distributed across groups without needing ground-truth labels (e.g., Predicted Prevalence, Predicted Positive Rate).
- Error-Based Metrics: Evaluates disparities in prediction errors using ground-truth labels, focusing on Type I errors (False Discovery Rate, False Positive Rate) and Type II errors (False Omission Rate, False Negative Rate).
- Disparity & Parity Assessment: Calculates disparities by comparing group metrics against a user-defined reference group (e.g., the majority group or a historically favored group). It applies a flexible parity threshold (such as the "80% rule", $\tau = 0.8$) to determine if a model passes a fairness test.
To bridge the gap between technical metrics and policy goals, the authors developed the "Fairness Tree", a conceptual navigation tool that guides users to the right metric based on whether their intervention is assistive or punitive.
Aequitas is highly accessible, offering deployment as a Python library, a command-line interface, or a web application tailored for non-technical stakeholders.
Datasets
The paper demonstrates Aequitas using three real-world public policy case studies alongside one public benchmark:
- Criminal Justice (Recidivism): Administrative data for 1.5 million individuals over 10 years (focusing on 400,000 with repeated interactions) to predict recidivism risk and match individuals with preventative interventions.
- Public Health (HIV Care Retention): Electronic health records of approximately 1,600 patients spanning 8 years (2008–2016) to predict the risk of dropping out of care.
- Public Safety and Policing: Over 20 million records spanning 15 years of police employee information and event records to build an Early Intervention System identifying officers at risk of adverse events.
- COMPAS Dataset: The publicly available Broward County recidivism dataset containing risk scores and 2-year outcomes for 7,214 individuals, used as a reproducible audit example in the appendix.
Key Insights and Ideas
- Context Dictates the Metric: Fairness isn't a one-size-fits-all mathematical formula. The choice of metric depends heavily on the nature of the intervention. For assistive programs (like housing aid), minimizing False Omissions (Type II errors) is paramount so vulnerable people aren't left behind. For punitive programs (like jail release decisions), minimizing False Positives (Type I errors) is critical to avoid unjust harm.
- Human-in-the-Loop Resource Constraints: Public policy problems often operate under strict resource limits, meaning models optimize for "precision at top $k$" (e.g., only having enough budget to help the top 150 highest-risk people) rather than overall accuracy or AUC-ROC. Aequitas explicitly accounts for this operational timeline.
- The Status Quo is Often Worse: While ML models frequently exhibit biases when audited, the case studies show they are often significantly less biased and far more accurate than the existing human heuristics or simplistic policy rules currently in production.
Key Results
- Criminal Justice: The selected model achieved 73% precision at the top 150 (outperforming the city's 43% heuristic baseline). The audit revealed a 2X False Omission Rate (FOR) disparity for young adults (18-25) and a 1.6X FOR disparity for African Americans compared to white individuals. However, it was still less biased overall than the current frequency-based heuristic.
- Public Health: The ML model improved prediction of patient drop-out by 20%–50% over expert clinician-generated rules. The audit showed that females were favorably biased (lower FOR), but the model reduced overall bias compared to the expert baseline.
- Public Safety: The early intervention model flagged 10%–15% more high-risk officers while reducing false positives by 40% compared to the existing system. The audit revealed unfavorable False Discovery Rate (FDR) bias for divorced and African American officers.
Limitations and Future Work
- Audit vs. Mitigation: Aequitas focuses primarily on auditing and detecting bias rather than automatically fixing or mitigating it.
- Human Decision Trade-offs: While the toolkit exposes disparities, it cannot make the final value judgment. It leaves the difficult ethical and political decisions of balancing accuracy vs. fairness to human policymakers.
- Future Directions: The authors emphasize the need to develop educational materials, workshops, and training to help both technical and non-technical audiences better understand the implications of algorithmic bias and make informed policy choices.
Summary updated