TL;DR
This paper presents the first comprehensive study of gender bias in Turkish word embeddings, revealing pervasive masculine defaults across syntax, semantics, and morphology. By analyzing embeddings trained on diverse Turkish corpora, the authors demonstrate how gender bias is embedded not only in word meanings but also in the very structure of the language, particularly through suffixes.
What Problem Does This Paper Address?
As large language models (LLMs) become increasingly integrated into our lives, understanding and mitigating the biases they inherit from training data is crucial. While much research has focused on gender bias in English word embeddings, similar investigations into other languages, especially those with rich morphology like Turkish, have been limited. This paper aims to fill this gap by providing a thorough analysis of gender bias in Turkish word embeddings, examining its manifestation across different linguistic dimensions and domains.
Methods
The authors employed static word embeddings, specifically Floret vectors, which incorporate subword information to better capture Turkish's agglutinative morphology. These embeddings were trained on three distinct Turkish corpora: a large web crawl (mC4), an academic corpus, and a medical corpus. To quantify gender bias, they adapted the method from Bolukbasi et al. (2016), calculating a gender bias score for each word based on its vector's proximity to "man" versus "woman" representations. This unsupervised approach allowed for an in-depth analysis of semantic associations, syntactic categories, and the impact of morphological suffixes on gender.
Datasets
The study utilized three Turkish datasets:
- mC4: A large web crawl corpus (76.4 million documents) serving as a broad reflection of societal language.
- Academic Crawl: A combined corpus of academic paper abstracts and full articles from Turkish sources (YÖK Açık Erişim and Dergipark), totaling over 645,000 documents.
- Medical Crawl: A corpus of medical research papers from Dergipark, comprising approximately 37,910 documents.
The paper does not introduce new datasets but rather uses existing ones for training and analysis.
Key Insights and Ideas
- Pervasive Masculine Defaults: Across all domains, Turkish word embeddings exhibit a strong tendency to associate words with masculinity, with a significant majority of vocabulary words leaning towards male associations.
- Morphology Matters: Turkish's rich morphology plays a crucial role in embedding gender. Suffixes, even at the subword level, can significantly influence or even flip the perceived gender of a word, a dimension often overlooked in bias studies.
- Domain-Specific Nuances: While masculine bias is prevalent everywhere, the specific semantic clusters associated with each gender vary by domain. For instance, feminine clusters in the web domain often relate to domesticity and appearance, while academic and medical domains show a slightly different distribution, though still biased.
- Syntactic Categories Reflect Bias: Gender bias is not confined to semantics; it extends to syntactic categories. Verbs, nouns, adjectives, and adverbs all show skewed gender distributions, with masculine associations dominating most categories.
- Societal Patriarchy Reflected: The authors connect their findings to the broader societal context of patriarchy in Turkey, suggesting that language reflects and reinforces these deeply ingrained cultural norms.
Key Results
- In the mC4 corpus, 87% of vocabulary words were associated with men, compared to 13% with women. This pattern was similar across academic (88% male) and medical (87% male) domains.
- Verbs and nouns predominantly carried masculine associations. For example, in the mC4 corpus, verbs were masculine 73.7% of the time, and nouns 73.7%.
- Profession names showed a stark gender bias, with only a few traditionally "feminine" professions appearing, while most, including leadership roles and tech company names, were masculine.
- The study identified specific suffixes that can alter word gender, such as the plural suffix "-lAr" which can shift masculine lemmas to feminine.
- The passive voice verbal suffix was found to change masculine verbs to feminine, potentially reflecting societal perceptions of women in passive roles.
Limitations and Future Work
The authors acknowledge that their study focuses on static word embeddings and does not capture the dynamic nature of contextual embeddings like BERT. They also note that while they filtered out explicit profanity, subtle forms of bias might still exist. Future work could involve investigating gender bias in contextual embeddings, exploring the impact of different subword strategies, and developing debiasing techniques specifically tailored for the Turkish language and its unique morphological characteristics. The paper also suggests further research into how these biases are amplified in downstream NLP applications.
Summary updated