It's All in the Name: Mitigating Gender Bias with Name-Based Counterfactual Data Substitution

Rowan Hall Maudslay, Hila Gonen, Ryan Cotterell, Simone Teufel

Abstract

This paper treats gender bias latent in word embeddings. Previous mitigation attempts rely on the operationalisation of gender bias as a projection over a linear subspace. An alternative approach is Counterfactual Data Augmentation (CDA), in which a corpus is duplicated and augmented to remove bias, e.g. by swapping all inherently-gendered words in the copy. We perform an empirical comparison of these approaches on the English Gigaword and Wikipedia, and find that whilst both successfully reduce direct bias and perform well in tasks which quantify embedding quality, CDA variants outperform projection-based methods at the task of drawing non-biased gender analogies by an average of 19% across both corpora. We propose two improvements to CDA: Counterfactual Data Substitution (CDS), a variant of CDA in which potentially biased text is randomly substituted to avoid duplication, and the Names Intervention, a novel name-pairing technique that vastly increases the number of words being treated. CDA/S with the Names Intervention is the only approach which is able to mitigate indirect gender bias: following debiasing, previously biased words are significantly less clustered according to gender (cluster purity is reduced by 49%), thus improving on the state-of-the-art for bias mitigation.

Introduction

Word embeddings are fundamental to modern Natural Language Processing, yet they frequently inherit and amplify societal biases present in their training data. A well-documented example is the stereotypical analogy man : computer programmer :: woman : homemaker. Addressing this requires moving beyond simple surface-level fixes.

Existing debiasing techniques generally fall into two categories:

  1. Projection-based methods (e.g., WED): These identify a "gender direction" in the vector space and attempt to neutralize it. However, this often fails to address "indirect bias"—the persistent clustering of words (like 'football' or 'business') that remain associated with a specific gender even after the primary direction is flattened.
  2. Counterfactual Data Augmentation (CDA): This method involves duplicating the training corpus and swapping gendered terms (e.g., "he" to "she"). While effective, it doubles the computational cost and can introduce statistical anomalies.

A critical oversight in both approaches is the role of first names. Names are powerful carriers of gendered associations, yet they are often ignored in debiasing pipelines.

Methodology: CDS and the Names Intervention

The authors introduce Counterfactual Data Substitution (CDS). Unlike CDA, which duplicates the corpus, CDS randomly substitutes gendered words with their counterparts with a 50% probability. This maintains the original corpus size and preserves more natural word frequency distributions.

Taxonomy of words within the embedding space, distinguishing between inherently gendered pairs and neutral words that may carry latent bias.

The core innovation is the Names Intervention. The authors argue that swapping pronouns while leaving gendered names (like "John") intact can actually reinforce bias. To solve this, they developed a name-pairing technique using a massive dataset from the US Social Security Administration.

Visualization of the name-pairing process, where names are matched based on their frequency (primary gender occurrences) and gender-specificity (secondary gender occurrences).

Using the Hungarian method for bipartite-graph matching, names are paired based on two criteria:

  • Frequency: Ensuring common names are paired with counterparts of similar commonality.
  • Gender-specificity: Matching names that are similarly "strictly" male or female based on their occurrence patterns.

Evaluation and Results

The methods were tested on the English Gigaword and Wikipedia corpora using benchmarks like SimLex-999 and WEAT.

Mitigating Indirect Bias

The Names Intervention proved to be the only method capable of significantly reducing indirect bias. By treating names, the researchers reduced "cluster purity"—a metric for how easily neutral words can be grouped by gender—by 49%.

Preserving Semantic Utility

Projection-based methods often act as a "sledgehammer," removing the very concept of gender and breaking useful analogies. In a "non-biased gender analogy" test (e.g., boy : girl :: nephew : niece), CDS variants outperformed projection methods by 19%, demonstrating that they preserve semantic relationships while removing prejudice.

Conclusion and Limitations

While CDS with the Names Intervention sets a new state-of-the-art for bias mitigation, several challenges remain. The current implementation relies on a gender binary, which does not account for non-binary identities. Additionally, linguistic complexities like the "his/her/hers" ambiguity and the cultural specificity of names (currently based on US data) suggest that future work must focus on localization and more complex, non-linear models of bias.

Summary updated

Abstract

This paper treats gender bias latent in word embeddings. Previous mitigation attempts rely on the operationalisation of gender bias as a projection over a linear subspace. An alternative approach is Counterfactual Data Augmentation (CDA), in which a corpus is duplicated and augmented to remove bias, e.g. by swapping all inherently-gendered words in the copy. We perform an empirical comparison of these approaches on the English Gigaword and Wikipedia, and find that whilst both successfully reduce direct bias and perform well in tasks which quantify embedding quality, CDA variants outperform projection-based methods at the task of drawing non-biased gender analogies by an average of 19% across both corpora. We propose two improvements to CDA: Counterfactual Data Substitution (CDS), a variant of CDA in which potentially biased text is randomly substituted to avoid duplication, and the Names Intervention, a novel name-pairing technique that vastly increases the number of words being treated. CDA/S with the Names Intervention is the only approach which is able to mitigate indirect gender bias: following debiasing, previously biased words are significantly less clustered according to gender (cluster purity is reduced by 49%), thus improving on the state-of-the-art for bias mitigation.
BibTeX
@inproceedings{maudslay2019it,
  author = {Rowan Hall Maudslay and Hila Gonen and Ryan Cotterell and Simone Teufel},
  title = {{It's All in the Name: Mitigating Gender Bias with Name-Based Counterfactual Data Substitution}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2019},
  url = {https://openreview.net/forum?id=yBSL_dVRMDs},
}
Figure