TL;DR
This paper identifies a new type of gender bias in Natural Language Inference (NLI) called "marked attribute bias," where models tend to assume "person" implies "man" by default. They propose a novel debiasing method, Multi-dimensional Information-weighted Soft Projection (MISP), which significantly reduces this bias without harming general embedding quality.
What Problem Does This Paper Address?
Word embeddings, the numerical representations of words used in NLP, are known to encode societal biases, including gender bias. Previous work has focused on debiasing these embeddings to mitigate issues like gender-stereotyped occupations in analogy tasks. However, this paper highlights that even after debiasing, a subtle but persistent bias remains in downstream tasks like Natural Language Inference (NLI). Specifically, they observe "marked attribute bias," where models infer that a generic term like "person" is more likely to be male (the "default" attribute) than female (the "marked" attribute). This bias can manifest even with implicit gender carriers like names or stereotypical occupations. Existing intrinsic bias measures and debiasing methods fail to adequately capture or address this specific type of bias, creating a gap in our understanding and mitigation strategies.
Methods
The authors introduce a new intrinsic bias measure called Multi-dimensional Information-weighted Direct Bias (MIDB). Unlike previous methods that often rely on a single "gender direction," MIDB considers a multi-dimensional gender subspace, defined using principal components derived from pairwise differences between gendered names. This measure showed a stronger correlation with the observed marked attribute bias than existing intrinsic measures.
Building on MIDB, they propose a novel debiasing scheme called Multi-dimensional Information-weighted Soft Projection (MISP). MISP projects word embeddings into the nullspace of this multi-dimensional gender subspace. Crucially, it uses a "soft" projection, where the extent of projection is weighted by the informativeness of each dimension in capturing gender information. This contrasts with "hard" projection methods that aim to completely remove bias.
Datasets
The paper introduces new test sets for evaluating marked attribute bias:
- Explicit gender words test set: Pairs of sentences where a premise about a "person" is followed by hypotheses using explicit gendered pronouns (e.g., "He prepared a meal," "She prepared a meal").
- Implicit gender carriers test set: Similar to the explicit set, but using names that are typically associated with a specific gender.
- Latent gender carriers test set: Utilizes stereotypical occupations (e.g., "doctor," "nurse") as latent carriers of gender bias.
They also evaluate their method on standard word similarity and analogy benchmarks, including RG, MTurk, MEN, SimLex999, Google, and MSR.
Key Insights and Ideas
- "Marked attribute bias" is a distinct and persistent form of gender bias: Even after applying existing debiasing techniques, NLI models continue to exhibit a tendency to default to male attributes when inferring from generic terms like "person."
- Existing intrinsic bias measures are insufficient: Measures like Direct Bias, Clustering, and Recoverability do not effectively predict or correlate with this marked attribute bias, highlighting a limitation in how we currently quantify embedding bias.
- A multi-dimensional gender subspace is more informative: Defining gender bias not as a single direction but as a multi-dimensional subspace, derived from gendered names, proves more effective in understanding and mitigating the bias.
- Soft projection is key: The "soft" nature of MISP, where projections are weighted, is crucial for its success, suggesting that a complete removal of gender information might not be necessary or even desirable for certain biases.
Key Results
- MISP significantly reduces marked attribute bias: When applied to GN-GloVe(wa) embeddings, MISP achieved a 26% error reduction on the explicit gender words test set compared to the previous best.
- Bias reduction is consistent across different bias types: The marked attribute bias was also significantly reduced on implicit gender carriers (names) and latent gender carriers (stereotypical occupations).
- No trade-off with general embedding quality: Applying MISP did not negatively impact performance on standard word similarity and analogy benchmarks. For instance, performance on the Google analogy task remained at 80.48% accuracy after MISP application.
- Existing debiasing methods fail to mitigate marked attribute bias: The paper shows that none of the analyzed existing debiasing methods (e.g., Hard Debias, GN-GloVe, Gender-Preserving) could successfully reduce the marked attribute error.
Limitations and Future Work
- Limited scope of gender subspace: The current definition of the multi-dimensional gender subspace relies on names. The authors suggest this approach could be extended to other attributes like race or ethnicity.
- Exploration of alternative weighting schemes: While MISP's soft projection with information weighting shows promise, further exploration of different weighting strategies and how they interact with learned gender subspaces is suggested.
- Investigating latent bias effects: The paper highlights the importance of understanding latent gender effects and suggests further investigation into their mitigation strategies.
- Generalizability of MISP: While MISP shows strong results on the marked attribute bias test set, its effectiveness on other downstream tasks and bias types warrants further study.
Summary updated