Mitigating Gender Bias in Machine Learning Data Sets

Susan Leavy, Gerardine Meaney, Karen Wade, Derek Greene

Abstract

Machine learning models often inherit societal biases from their training data. This paper proposes a diagnostic framework grounded in feminist linguistics to identify these biases upstream. By analyzing 19th-century fiction and a decade of modern journalism, the authors reveal how linguistic structures like binomial ordering and occupational premodification serve as measurable indicators of gender ideology.

Introduction

The quality of machine learning models is fundamentally limited by the data they consume. While post-training "de-biasing" is common, implicit biases often persist. This research argues for a shift "upstream" to systematically evaluate training data itself. Detecting gender ideology is challenging as it is deeply embedded in language; simple keyword searches are insufficient to capture complex sociolinguistic nuances.

Methodology

The authors developed a framework combining Natural Language Processing (NLP) with qualitative sociolinguistic theory to identify bias-encoding features:

  • Word Embeddings: A 100-dimensional Word2Vec model was used to analyze conceptual relationships via cosine distances.
  • Thematic Lexicons: Seed terms were expanded using embedding similarities and General Inquirer dictionaries to categorize words into themes such as "emotion" or "family."
  • Linguistic Feature Extraction:
    • Premodification: Identifying gendered adjectives added to neutral nouns (e.g., "female doctor"), signaling one gender as the "exception."
    • Binomial Ordering: Analyzing pairs like "men and women" vs. "women and men," where the first position traditionally denotes the "default" or "powerful" entity.
  • Androcentric Generics: Tracking terms like "mankind" used as neutral defaults.

Datasets

The study compares two distinct corpora:

  1. British Library Digital Corpus: 16,426 volumes of 19th-century fiction, providing a baseline for historical gender stereotypes.
  2. The Guardian (2009-2018): A contemporary corpus of online articles, allowing for the tracking of modern discourse and feminist influence.

Key Findings

The Persistence of the "Default Male"

In historical texts, "female" appeared 2.5 times more often than "male" because "male" was the assumed default. In modern data, there is an increase in gender-premodified terms like "female empowerment." While often used in critical contexts, ML models may still learn underlying associations (e.g., "fragility" with "female") without grasping the critique.

Power in Syntax

Binomial ordering remains a stubborn indicator of power. In 19th-century fiction, men were listed first in 87% of pairs. In The Guardian, this figure remained high at 78% in 2009 and 74% in 2018, suggesting a persistent societal conception of male as the primary subject.

Representation and Occupational Bias

The representation of women in text remains disproportionately low. In 2009, only 20% of gendered pronouns in The Guardian were female, rising to 30% by 2018—still lagging behind the levels found in 19th-century fiction.

Proportion of pronoun mentions across corpora and over time.

Furthermore, the variety of occupations premodified by gender has expanded significantly in modern journalism, reflecting a heightened focus on gender gaps but also reinforcing the "markedness" of certain roles.

Gender-premodified occupational terms in historical and modern corpora.

Conclusion

This framework serves as a vital diagnostic tool for identifying bias in ML training pipelines. Future work will focus on scaling these metrics to broader corpora, developing contextual awareness to distinguish between perpetuating and critiquing stereotypes, and integrating these insights into automated "gender-proofing" techniques.

Summary updated

Abstract

Artificial Intelligence has the capacity to amplify and perpetuate societal biases and presents profound ethical implications for society. Gender bias has been identified in the context of employment advertising and recruitment tools, due to their reliance on underlying language processing and recommendation algorithms. Attempts to address such issues have involved testing learned associations, integrating concepts of fairness to machine learning and performing more rigorous analysis of training data. Mitigating bias when algorithms are trained on textual data is particularly challenging given the complex way gender ideology is embedded in language. This paper proposes a framework for the identification of gender bias in training data for machine learning.The work draws upon gender theory and sociolinguistics to systematically indicate levels of bias in textual training data and associated neural word embedding models, thus highlighting pathways for both removing bias from training data and critically assessing its impact.
BibTeX
@inproceedings{leavy2020mitigating,
  author = {Susan Leavy and Gerardine Meaney and Karen Wade and Derek Greene},
  title = {{Mitigating Gender Bias in Machine Learning Data Sets}},
  booktitle = {Bias},
  year = {2020},
  doi = {10.1007/978-3-030-52485-2\_2},
}
Figure