TL;DR
This paper tackles the pervasive issue of gender bias in text by proposing a new taxonomy and a methodology to identify a specific subtype: Gender Generalization. They introduce a novel filtering approach to create a dataset of these biased sentences, aiming to train models that can detect and potentially correct such biases at the source.
What Problem Does This Paper Address?
Gender bias in text is a significant problem, especially in the age of machine learning where algorithms can learn and amplify these biases from training data. While much research has focused on debiasing existing models, this paper argues for a dataset-level approach. They identify a gap in quantitative studies that specifically aim to build datasets for detecting gender bias, particularly focusing on the "Gender Generalization" subtype, where gender-neutral terms are incorrectly associated with gender-specific pronouns. This bias, if unchecked, can perpetuate harmful stereotypes and inequalities.
Methods
The authors propose a taxonomy to categorize gender bias in text, distinguishing between structural and contextual biases. Their primary focus is on Gender Generalization, a structural bias where a gender-neutral noun is referred to by a gender-exclusive pronoun (e.g., "A programmer must always carry his laptop").
To identify these sentences, they developed a multi-stage filtering pipeline:
- Coreference Resolution: Using AllenNLP's pre-trained model, they identify mentions that refer to the same entity within a sentence.
- Gendered Pronoun Filter: Sentences containing gendered pronouns (he, him, his, etc.) are flagged.
- Human Name Filter: Sentences with identified human names are removed, as these are not considered Gender Generalization (e.g., "Jason must not abandon the place where he was brought up" is not a generalization). They use Named Entity Recognition (NER) for this.
- Gendered Term Filter: Sentences containing explicitly gendered terms (e.g., "policemen," "actress") are removed. This filter uses WordNet and the Lesk algorithm for word sense disambiguation.
- Pronoun Link Filter: Sentences where coreference clusters only contain pronouns (and no gender-neutral human entity) are removed, as they don't fit the definition of generalization.
The filtered sentences were then sent for human labeling via crowdsourcing on Figure Eight.
Datasets
The paper utilizes two main sources for their study:
- IMDB Movie Review Dataset: This dataset was used as a starting point but was found to contain too much noise (person-specific mentions) for effectively identifying Gender Generalization.
- Project Gutenberg Corpora: Nine different corpora from Project Gutenberg were used, including texts like "Business Hints for Men and Women," "Magna Carta," and "Scouting For Girls." These were chosen for their general applicability and potential to contain Gender Generalization sentences.
The authors also created a newly labeled dataset of Gender Generalization sentences, which is a key contribution of the paper.
Key Insights and Ideas
- Dataset-level bias detection is crucial: The paper emphasizes that addressing gender bias at the dataset level, rather than solely through model debiasing, is a more fundamental approach.
- Gender Generalization is subtle but prevalent: While not always obvious, this specific type of bias can be systematically identified through syntactic analysis.
- Corpus selection matters: The choice of text source significantly impacts the ability to find relevant examples of bias. General texts (like those from Project Gutenberg) are more suitable for identifying broad societal gender generalizations than highly specific texts (like movie reviews).
- Human validation is indispensable: Even with sophisticated filters, human judgment is necessary to confirm the presence and subtype of gender bias, especially for nuanced cases.
Key Results
- After filtering, 808 out of 1627 sentences from Project Gutenberg corpora were labeled as Gender Generalization bias, accounting for approximately 49.7% of the filtered data.
- The overall inter-rater reliability among human labelers was 61.14%.
- An evaluation of a random subset of labeled sentences showed an F1 score of 73.9% and an accuracy of 75%.
- The study found that Gender Generalization sentences represented a small fraction (below 5%) of each corpus explored.
Limitations and Future Work
- Filter Quality: The authors acknowledge that the quality of the pre-trained models used in their filters can impact the retained sentences, and improving these filters is a direction for future work.
- Labeler Agreement: The inter-rater reliability was moderate, suggesting that some sentences were ambiguous or difficult to classify, even for human labelers.
- Scope of Bias: The current work focuses on only one subtype of gender bias (Gender Generalization). Future work aims to expand the taxonomy and develop methods to capture other types of biases (societal, behavioral, contextual).
- Dataset Expansion: The authors plan to create a more comprehensive dataset encompassing various gender biases and explore more advanced filtering and classification mechanisms. They also suggest exploring different data presentation methods for labeling to potentially improve labeler accuracy and efficiency.
Summary updated