Assessing and Enforcing Fairness in the AI Lifecycle

Roberta Calegari, Gabriel G. Castañé, Michela Milano, Barry O'SullivanIJCAI 2023 · DOI

TL;DR

This paper provides a comprehensive survey of fairness in AI, systematically mapping different fairness notions and mitigation techniques to their appropriate stages within the AI lifecycle. It aims to equip AI practitioners with a structured understanding to proactively address bias, rather than reactively fixing it, thereby fostering more responsible AI development.

What Problem Does This Paper Address?

The increasing integration of AI systems into critical decision-making processes (e.g., banking, healthcare, legal) highlights a significant challenge: ensuring these systems are fair and do not perpetuate or amplify societal biases. While a wealth of literature exists on AI fairness, it's often fragmented, making it difficult for developers to navigate the landscape of fairness metrics and mitigation techniques. This paper addresses the gap in understanding when and how to apply fairness considerations throughout the entire AI development lifecycle, from data collection to model deployment and monitoring. The authors argue that a proactive, lifecycle-aware approach is crucial for effectively building and maintaining fair AI systems, moving beyond ad-hoc solutions.

Methods

The paper's core contribution lies in its systematic categorization and mapping of fairness concepts. It organizes fairness into two main activities within the AI lifecycle:

  • Fairness Awareness (Assessment): This involves understanding and measuring bias. The paper categorizes fairness notions into two broad types:
  • Procedural Fairness: Focuses on the fairness of the decision-making process itself, ensuring equal treatment.
  • Outcome Fairness: Focuses on the fairness of the results of the decision-making process. This is further broken down into:
  • Observational Fairness: Based on statistical relationships between features, outcomes, and sensitive attributes (e.g., independence, separation, sufficiency).
  • Causal Fairness: Utilizes causal inference to understand how sensitive attributes influence outcomes, allowing for more nuanced analysis (e.g., counterfactual fairness).
  • Fairness Reparation/Mitigation (Enforcement): This involves techniques to reduce or eliminate bias. The paper maps these techniques to three phases of the AI lifecycle:
  • Pre-processing: Modifying the data before training (e.g., blinding, adversarial learning, causal approaches, relabelling, re-sampling, re-weighting).
  • In-processing: Incorporating fairness directly into the model training process (e.g., adversarial learning, constraint optimization, regularization).
  • Post-processing: Adjusting the model's output after training (e.g., calibration, relabelling, thresholding).

The paper's novelty lies in its structured approach, creating a framework that connects specific fairness notions with concrete intervention techniques and their placement within the AI lifecycle.

Datasets

Not discussed in the paper. The paper focuses on a conceptual survey and framework rather than empirical evaluation on specific datasets.

Key Insights and Ideas

  • Fairness is not a one-size-fits-all solution: Different fairness notions (e.g., independence vs. separation) are often mutually exclusive and context-dependent, requiring careful selection based on legal, ethical, and social considerations.
  • Proactive vs. Reactive Fairness: Addressing fairness early in the AI lifecycle (pre-processing) is generally more effective and less disruptive than attempting to fix bias in deployed models (post-processing).
  • The AI Lifecycle as a Fairness Map: The paper's central insight is that understanding the AI lifecycle provides a crucial framework for deciding when and how to intervene to ensure fairness. Each stage (pre-processing, in-processing, post-processing) offers different opportunities and challenges for fairness enforcement.
  • Trade-offs are inevitable: Achieving fairness often involves trade-offs with other desirable AI metrics like accuracy and performance. The paper highlights the need to balance these competing requirements.
  • "Fairness through unawareness" is insufficient: Simply removing sensitive attributes from data can be ineffective, as biases can persist through proxy variables.

Key Results

The paper does not present quantitative results from experiments. Instead, its key "results" are conceptual:

  • A structured framework for understanding and categorizing fairness notions and mitigation techniques.
  • A mapping of these techniques to the distinct phases of the AI lifecycle.
  • Identification of the advantages and disadvantages of applying fairness interventions at different stages.
  • A comprehensive overview of existing fairness literature, organized by the proposed framework.

Limitations and Future Work

Limitations:

  • Educational Gap: A significant challenge is the lack of AI practitioners' awareness and understanding of fairness, leading to it being perceived as a burden rather than an integral part of development.
  • Methodological Gap: There's a need for more standardized methodologies to tackle fairness across different AI lifecycle stages, especially for developers working on specific problems.
  • Technical Diversification: Current research often focuses on supervised learning and binary classification. More work is needed on other learning paradigms and with more realistic, representative datasets.
  • Interpretability and Transparency: The need for interpretable and transparent AI systems is crucial for building user trust and is often legally mandated. Causality-based approaches are highlighted as promising for improving explainability.
  • Metric Balancing: Fairness metrics need to be balanced between individual and group notions, and current research often prioritizes group fairness due to lower development effort and computational cost.
  • Experimentation Environments: The lack of accessible environments for testing and comparing different fairness notions and techniques hinders progress.

Future Work:

  • Development of fairer algorithms and, more importantly, procedures to reduce bias in data, potentially by integrating humans and algorithms in the decision workflow.
  • Further research into causality-based approaches for better understanding and explaining unfairness.
  • Creation of robust experimentation environments to facilitate the testing and comparison of fairness interventions.
  • Strengthening existing fairness metrics and developing new ones to address a wider range of fairness concerns.
  • Addressing the educational aspect by providing clear guidance and methodologies for AI practitioners.

Summary updated

BibTeX
@inproceedings{calegari2023assessing,
  author = {Roberta Calegari and Gabriel G. Castañé and Michela Milano and Barry O'Sullivan},
  title = {{Assessing and Enforcing Fairness in the AI Lifecycle}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2023},
  doi = {10.24963/ijcai.2023/735},
}
Figure