DiFair: A Benchmark for Disentangled Assessment of Gender Knowledge and Bias

Mahdi Zakizadeh, Kaveh Eskandari Miandoab, Mohammad Taher PilehvarEMNLP 2023 · DOI

Introduction

The pursuit of fairness in Pretrained Language Models (PLMs) has largely focused on "blind debiasing"—techniques designed to make models gender-neutral. However, this approach often overlooks a critical dimension: gender knowledge. While we want a model to be neutral for a prompt like "The [MASK] is a doctor," we require it to be factually accurate for "The Queen spent [MASK] life in London." Current debiasing methods risk "lobotomizing" models, stripping away factual understanding in the name of neutrality. This paper introduces DIFAIR, a benchmark designed to disentangle gender bias from gender knowledge, addressing the inherent fairness-performance trade-off.

Methodology: Disentangling Bias and Knowledge

The researchers frame gender assessment as a Masked Language Modeling (MLM) task. To quantify a model's performance, they propose the Gender Invariance Score (GIS), defined as the harmonic mean of two distinct metrics:

  1. Gender-Neutral Score (GNS): Measures neutrality in contexts where gender is not specified (e.g., "The [MASK] is a musician").
  2. Gender-Specific Score (GSS): Measures accuracy in contexts where gender is determined by facts or linguistic cues (e.g., "Marie Curie's [MASK] work").
Figure 1: Illustration of the calculation for Gender-Neutral Score (GNS) and Gender-Specific Score (GSS) using a bidirectional language model.

By using the harmonic mean, the GIS penalizes models that are either highly biased or factually ignorant. A high GIS requires a balance of both neutrality and knowledge.

The DIFAIR Dataset

The DIFAIR dataset consists of 2,506 manually curated instances from English Wikipedia and Reddit. It is categorized into:

  • Gender-Specific (984 instances): Sentences where gender is fixed by historical facts, biological traits, or coreference.
  • Gender-Neutral (1,522 instances): Sentences where any gender is equally valid.

To ensure models rely on contextual understanding rather than simple memorization, the authors employed anonymization, replacing specific names and dates with tokens during testing.

Key Insights: The Impact of Scale and Debiasing

The study reveals several critical trends in how models handle gender:

  • The Distillation Paradox: While distilling large models (e.g., DistilBERT) improves neutrality, it significantly degrades gender knowledge—often by over 25%.
  • Size vs. Fairness: Larger models like XLNet-large possess superior gender knowledge but often exhibit higher baseline bias compared to smaller models.
  • Temporal Bias: Models act as "temporal snobs," showing higher bias in historical contexts (e.g., the 1800s) than in modern ones, reflecting the biases of their training corpora.
  • Debiasing Trade-offs: Techniques like Counterfactual Data Augmentation (CDA) effectively boost GNS but can cause a collapse in GSS, as shown in the comparison between standard and CDA-augmented BERT models.
Figure 4: Performance trends across different historical periods for BERT-large, BERT-large with CDA, and ALBERT-large, highlighting the fluctuations in gender knowledge and neutrality.

Results and Performance Gap

There remains a significant gap between AI and human performance. While humans achieve a GIS of 93.85%, the top-performing model, XLNet-large, reached only 70.01%. Applying CDA to BERT-base improved its neutrality (GNS) from 63.91 to 86.44 but simultaneously crashed its knowledge score (GSS) from 58.02 to 34.05. Similarly, DistilBERT achieved high neutrality (93.33) but failed on knowledge (29.67), resulting in a low overall GIS of 45.02.

Conclusion

The DIFAIR benchmark highlights the necessity of moving beyond "blind" debiasing. Future fairness interventions must be "knowledge-preserving," ensuring that models remain factually competent while shedding unfair prejudices. As the field progresses, expanding this framework to non-binary gender identities and diverse languages will be essential for creating truly robust and fair AI systems.

Summary updated

BibTeX
@inproceedings{zakizadeh2023difair,
  author = {Mahdi Zakizadeh and Kaveh Eskandari Miandoab and Mohammad Taher Pilehvar},
  title = {{DiFair: A Benchmark for Disentangled Assessment of Gender Knowledge and Bias}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2023},
  doi = {10.18653/v1/2023.findings-emnlp.127},
}
Figure