TL;DR
This paper tackles the challenge of reliably comparing bias mitigation algorithms in word embeddings by proposing a standardized methodology that controls for word set selection and vector normalization. Their empirical study reveals that this unified approach leads to more consistent results, highlighting the importance of rigorous experimental design for fair algorithm evaluation.
What Problem Does This Paper Address?
Word embeddings (WE), which represent words as dense vectors, have been shown to inadvertently capture societal biases (e.g., gender, racial stereotypes) present in the training data. To combat this, researchers have developed two main types of solutions: (1) metrics to quantify bias (like WEAT) and (2) algorithms to reduce bias (like Hard Debias). However, comparing these mitigation algorithms has been problematic due to inconsistencies in how they are evaluated. Specifically, previous studies often:
- Use different sets of words when applying mitigation algorithms and when evaluating bias.
- Have overlap between the words used for learning mitigation and those used for evaluation, potentially leading to inflated performance.
- Apply vector normalization inconsistently, which can affect bias measurements.
These confounding factors make it difficult to determine which mitigation algorithms are truly effective and to compare them reliably. This paper aims to address this by proposing a simple, yet rigorous, methodology to standardize the comparison process.
Methods
The authors propose a methodology to standardize the comparison of bias mitigation algorithms by addressing three key issues:
- Word Set Standardization: They enforce the use of identical word sets for both bias measurement metrics and mitigation algorithms. This involves carefully defining and constraining the relationships between "target" words (representing social groups), "attribute" words (representing traits/professions), "bias definition" words (used by algorithms to learn bias direction), "gender-specific" words (inherently gendered and excluded from mitigation), and "objective" words (the rest of the vocabulary).
- Managing Word Set Overlap: They implement constraints to prevent problematic overlaps between these word sets. For instance, target words should not overlap with objective words, and bias definition words should not overlap with attribute words. This separation is analogous to the training/testing split in supervised learning, ensuring that evaluation is not biased by the training data.
- Controlling Vector Normalization: Recognizing that vector normalization can impact bias metrics, they propose two approaches for fairer comparison: either normalize the word embedding model before applying algorithms that don't normalize, or reverse the normalization applied by algorithms like Hard Debias and RAN after mitigation.
Datasets
The study utilizes the glove-wiki-gigaword-300 word embedding model, a widely used model in the field.
For evaluation, they employ six bias measurement metrics:
- WEAT (Caliskan et al., 2017)
- WEAT ES (Caliskan et al., 2017)
- RND (Garg et al., 2018)
- RNSB (Sweeney and Najafian, 2019)
- ECT (Dev and Phillips, 2019)
- RIPA (Ethayarajh et al., 2019)
Four bias mitigation algorithms are used:
- Hard Debias (HD) (Bolukbasi et al., 2016)
- Double Hard Debias (DHD) (Wang et al., 2020)
- Repulsion Attraction Neutralization (RAN) (Kumar et al., 2020)
- Half Sibling Regression (HSR) (Yang and Feng, 2020)
The word sets used for these metrics and algorithms are drawn from existing literature and standardized according to the proposed methodology.
Key Insights and Ideas
- The Devil is in the Details: Inconsistencies in word set selection and vector normalization are significant confounding factors that can obscure the true performance of bias mitigation algorithms.
- Standardization Breeds Consistency: When a standardized methodology is applied, the performance gap between different bias mitigation algorithms narrows, suggesting they are more comparable than previously thought.
- Holistic Approach is Key: While standardizing word sets, managing overlap, and controlling normalization individually offer some benefits, their combined effect is crucial for achieving a statistically significant reduction in variability and a more objective evaluation.
- Bias Mitigation is Not a Silver Bullet: The study implies that even with improved evaluation, the inherent challenges of bias in language data mean that mitigation is an ongoing effort.
Key Results
The paper presents a comprehensive comparison of algorithms under their baseline and proposed methodologies.
- Baseline vs. Proposed: In the original setting, HD and RAN performed significantly better than DHD and HSR. However, with the proposed methodology (both normalization approaches), the performance gap between algorithms reduced, and DHD showed notable improvement.
- Reduced Variability: The proposed methodology consistently led to a reduction in the standard deviation of metric changes across algorithms. For instance, when reversing normalization, the average standard deviation across metrics decreased from 0.162 to 0.053 (p-value 0.04), indicating more consistent bias reduction.
- Component Analysis:
- Standardizing word sets alone showed a reduction in variability (p=0.16), but not statistically significant.
- Managing overlap between sets also reduced variability (p=0.23), but less so than the full methodology.
- Focusing solely on vector normalization had a minimal, non-statistically significant impact (p=0.92) on overall variability.
- Figure 1 visually demonstrates that all components contribute to reducing variability, but the full methodology achieves the most substantial and statistically significant reduction.
Limitations and Future Work
- Binary Gender Assumption: The study acknowledges that using gender as a binary variable is a simplification and a sensitive issue, noting their main objective was to replicate previous research. They recognize the need to review and update this approach.
- Limited Scope: The current work focuses on gender bias and specific types of word embeddings.
- Future Directions: The authors plan to extend their methodology to contextualized embeddings and large language models. They also aim to expand their work to diverse languages and explore various forms of bias beyond gender.
Summary updated