FROC: Building Fair ROC from a Trained Classifier

Avyukta Manjunatha Vummintala, Shantanu Das, Sujit GujarAAAI 2025 · DOI

TL;DR

This paper introduces FROC, a novel post-processing technique to make existing, potentially biased classifiers fair across different protected groups without retraining. FROC achieves this by transforming the classifier's output to ensure that its Receiver Operating Characteristic (ROC) curves remain close for all groups, minimizing accuracy loss.

What Problem Does This Paper Address?

In today's world, machine learning models are increasingly used for critical decisions like college admissions, loan applications, and even criminal justice. However, these models can inadvertently discriminate against certain groups based on protected attributes like gender, race, or age. While various fairness metrics and methods exist, many require retraining the entire model, which is often impractical or impossible for deployed systems. Furthermore, existing fairness metrics often focus on a single decision threshold, failing to address fairness across the entire spectrum of possible thresholds a practitioner might choose. This paper tackles the challenge of achieving fairness in score-based classification after a model has already been trained, ensuring that the model's predictions are fair regardless of the chosen decision threshold, and with minimal impact on overall performance.

Methods

The core of this paper is the FROC (Fair ROC) algorithm, a post-processing method designed to transform a potentially unfair classifier's scores into fair ones. The approach is built upon the following key ideas:

  • ε-Equalized ROC: This is a new fairness notion introduced by the authors. It requires that for any chosen decision threshold, the False Positive Rates (FPRs) and True Positive Rates (TPRs) for different protected groups are "close" to each other, specifically within an ε distance in the Lp norm for their respective ROC curves. This ensures fairness across all possible thresholds.
  • ROC Transport: FROC operates by "transporting" the ROC curve of the disadvantaged group towards the ROC curve of the advantaged group. This is achieved by making queries to the ROCs of both groups and then using a piece-wise linear approximation (PLA) of these ROCs.
  • Geometric Approach: The algorithm uses a geometric interpretation of ROC curves. It identifies points on the ROC curve of the advantaged group that are within the ε-distance boundary of the disadvantaged group's ROC. The algorithm then strategically "shifts" points from the advantaged group's ROC (referred to as ROCup) to be closer to the disadvantaged group's ROC (ROCdown). This involves operations like "CutShift," "UpShift," and "LeftShift" based on geometric relationships and area calculations.
  • Minimizing AUC Loss: A crucial aspect of FROC is its aim to minimize the loss in Area Under the Curve (AUC), a standard measure of classifier performance. The authors provide theoretical guarantees that under certain conditions, FROC achieves optimal fairness with minimal AUC degradation.

Datasets

The paper evaluates FROC on the following benchmark datasets:

  • ADULT: A widely used dataset for demographic prediction.
  • COMPAS: A dataset commonly used for recidivism prediction.

The authors also mention additional experiments on the CelebA dataset in the appendix.

Key Insights and Ideas

  • Fairness Across All Thresholds: The introduction of ε-Equalized ROC is a significant contribution, moving beyond single-threshold fairness notions to guarantee fairness across the entire decision-making spectrum. This is particularly valuable for practitioners who may adjust thresholds based on operational needs.
  • Post-Processing Power: The paper demonstrates that effective fairness interventions can be achieved after model training, without the need for costly retraining. This makes fairness solutions more accessible for real-world deployment.
  • Geometric Intuition for Fairness: FROC's geometric approach to "transporting" ROC curves offers an intuitive way to understand how to balance fairness and performance. The visual representation of ROC curves and their manipulation provides a clear conceptual framework.
  • Trade-off Management: The explicit goal of minimizing AUC loss while achieving fairness highlights the practical challenge of balancing these two often competing objectives. FROC aims to find an optimal sweet spot.

Key Results

  • Improved Fairness: FROC consistently improves fairness metrics, with experiments showing up to a 7-8% improvement in fairness.
  • Minimal Accuracy Drop: The post-processing with FROC results in a minimal drop in accuracy, often less than 2%, and in some cases, less than 0.1% for specific ε values.
  • Competitive Performance: In some scenarios, FROC's performance (in terms of accuracy) can even surpass that of in-processing methods, especially when those methods might overachieve fairness targets at the cost of significant accuracy reduction.
  • Theoretical Guarantees: The paper provides theoretical analysis and proofs demonstrating that FROC achieves optimal fairness under specific assumptions and minimizes AUC loss.

Limitations and Future Work

  • Assumptions for Optimality: The theoretical optimality guarantees of FROC are dependent on certain assumptions, such as the ROC curves intersecting the norm boundary at most twice. The paper notes that FROC remains operational even if these assumptions are violated, but optimality might not be guaranteed.
  • ROC Intersection: The current approach assumes that ROC curves do not intersect or intersect in regions that practitioners typically avoid (FPR ≤ 0.2 or TPR > 0.5). Addressing scenarios with intersecting ROCs in critical regions is left for future work.
  • Lp Norm: The current formulation uses the Lp norm for measuring distance between ROCs. The paper suggests exploring different distance metrics for fairness.
  • Performance Measures: Future work could also focus on optimizing for different performance measures beyond AUC.

Summary updated

BibTeX
@inproceedings{vummintala2025froc,
  author = {Avyukta Manjunatha Vummintala and Shantanu Das and Sujit Gujar},
  title = {{FROC: Building Fair ROC from a Trained Classifier}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i25.34836},
}
Figure