Measuring Gender Bias in Word Embeddings across Domains and Discovering New Gender Bias Word Categories

Kaytlin Chaloner, Alfredo MaldonadoACL 2019

TL;DR

This paper investigates how gender bias manifests in word embeddings trained on diverse data sources, revealing that bias levels and types vary significantly by domain. They introduce a novel unsupervised method for discovering new gender bias categories by clustering word embeddings, offering a more comprehensive understanding of these biases beyond pre-defined categories.

What Problem Does This Paper Address?

Artificial intelligence (AI) systems, particularly those in Natural Language Processing (NLP), are increasingly used for decision-making. However, these systems can inherit and amplify societal biases present in their training data, leading to unfair outcomes. Gender bias in word embeddings, a fundamental component of many NLP models, is a significant concern. While previous work has established the existence of gender bias in embeddings and proposed methods like the Word Embedding Association Test (WEAT) to detect it, there's a lack of comprehensive studies examining this bias across different domains and exploring new ways to identify bias categories beyond established ones. This paper aims to fill this gap by analyzing gender bias in embeddings from diverse domains and proposing a method to discover novel bias categories.

Methods

The authors employ the Word Embedding Association Test (WEAT), a statistical method adapted from psychology, to measure gender bias. WEAT uses cosine similarity to assess the association between target word sets (e.g., "career" words) and attribute word sets representing gender concepts (e.g., "male," "female"). They use a permutation test to determine the statistical significance of the observed bias.

For discovering new bias categories, they introduce an unsupervised clustering-based method. This involves:

  1. Clustering word embeddings within a given embedding set using K-Means++.
  2. For each cluster, identifying the most male- and female-associated words based on their similarity to the attribute word sets (using a measure similar to WEAT's association score).
  3. These lists of words are then considered as potential new gender bias categories.

Datasets

The paper uses four sets of word embeddings, each trained on corpora from a different domain:

  • Google News: Embeddings trained on the Google News corpus (reproduced for validation against prior work).
  • Twitter: Embeddings trained on a large corpus of Twitter micro-posts.
  • PubMed: Embeddings trained on biomedical literature (PMC and PubMed).
  • GAP (Gendered Ambiguous Pronouns): A gender-balanced corpus derived from Wikipedia, specifically designed to be balanced with respect to gendered pronouns.

Key Insights and Ideas

  • Domain Matters: The level and type of gender bias in word embeddings are not uniform; they are heavily influenced by the domain of the training data. Biomedical corpora, for instance, show less bias than news or social media.
  • WEAT's Limitations: While WEAT is effective for known bias categories, it struggles when target or attribute words are not present in the embedding's vocabulary (out-of-vocabulary words), as seen with the GAP corpus. This highlights the need for methods that leverage the embedding's own vocabulary.
  • Discovering New Bias Categories: The proposed clustering method successfully identifies thematically cohesive groups of gender-associated words, offering a way to expand the limited set of well-established bias categories.
  • Potential for Harmful vs. Harmless Bias: The discovered categories sometimes mix genuinely harmful gender biases with more innocuous associations (e.g., names of people). This suggests a need for further filtering or classification to distinguish between them.

Key Results

  • Google News: Showed statistically significant gender bias across all five tested categories (career vs. family, math vs. arts, science vs. arts, intelligence vs. appearance, strength vs. weakness).
  • Twitter: Exhibited bias primarily in the "career vs. family" category, with other categories showing weaker associations.
  • PubMed: Demonstrated the least gender bias, as expected for a scientific domain, though some bias was still detected.
  • GAP: Showed some bias, but the results were less conclusive due to a high number of out-of-vocabulary words, underscoring the importance of domain-specific vocabulary.
  • The newly discovered bias categories (from clustering) consistently showed statistically significant bias (p-value < 0.001) with high effect sizes across all tested embedding sets.

Limitations and Future Work

  • Out-of-Vocabulary Words: The WEAT method's effectiveness is limited by the presence of target and attribute words in the embedding's vocabulary, particularly impacting smaller or specialized corpora like GAP.
  • Distinguishing Harmful Bias: The unsupervised discovery method identifies gender-associated word groups, but further work is needed to differentiate between harmless associations (like names) and genuinely harmful gender biases.
  • Future Directions:
    • Develop a classifier to distinguish between harmless and harmful gender-biased words.
  • Involve domain experts and leverage existing lexical resources (like WordNet) and ontologies to validate discovered categories.
  • Investigate the impact of discovered bias categories on NLP system performance.
  • Further explore the influence of hyperparameters (like window size) on bias in embeddings.
  • Extend the analysis to other languages and domains.

Summary updated

BibTeX
@inproceedings{chaloner2019measuring,
  author = {Kaytlin Chaloner and Alfredo Maldonado},
  title = {{Measuring Gender Bias in Word Embeddings across Domains and Discovering New Gender Bias Word Categories}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2019},
}
Figure