TL;DR
This paper investigates gender bias in Wikipedia article titles across four languages, revealing that only half of gender-specific articles have a symmetrical counterpart for the other gender. This imbalance, particularly prevalent in sports and social topics, highlights a systematic issue that can propagate biases in NLP models.
What Problem Does This Paper Address?
While previous research has focused on gender bias within Wikipedia biographies, this paper addresses the broader question of how gender is represented in article titles across all domains. Existing NLP systems trained on Wikipedia data can inherit and amplify these biases, leading to representational harms. For instance, a search engine might default to assuming a "sports person" is male if male-associated articles are more prevalent or lack explicit gender markers. This study aims to quantify and understand these title-level inequalities to inform debiasing strategies.
Methods
The authors developed a three-step heuristic method to identify concept-related article tuples that exhibit gender imbalance in their titles.
- Title Filtering: They first filtered Wikipedia titles using manually selected gender-indicating words (e.g., "women," "men," "female," "male") and their inflected forms for German, Polish, and Turkish. They intentionally excluded singular forms like "woman" and "man" in some languages due to their frequent use in proper names.
- Meta-Categorization: Each filtered article was assigned one of five meta-categories: Sports, Lists, Social, Names, or Other. This categorization was based on keywords and Wikipedia categories.
- Grouping: Articles were then grouped into concept-related tuples. The primary goal was to find pairs of "Women" and "Men" articles describing the same concept by matching titles after removing gender indicators. Generic articles were also incorporated to form "Women | Men | Generic" tuples. Tuples were excluded if they belonged to the "Names" meta-category to avoid pairing proper names.
Datasets
The study analyzed Wikipedia editions in four languages: Turkish, English, German, and Polish. No new datasets were introduced; the research utilized existing Wikipedia content.
Key Insights and Ideas
- Ubiquitous Imbalance: Gender bias in Wikipedia titles is not limited to biographies but is widespread across various topics.
- Symmetry is the Exception: Only about half of the gender-related articles have a symmetrical counterpart for the other gender.
- Male Generic Bias: The "W|G" (Women | Generic) tuple type is significantly more frequent than "M|G" (Men | Generic), suggesting a tendency to treat male as the default or generic concept.
- Sports and Social Topics Dominate: Sports and social-related issues are the most affected domains by these title inequalities.
- Implicit Bias in Naming: Even when titles appear symmetrical, subtle biases can exist, such as the omission of "men's" in team names (e.g., "England national football team" referring to the men's team).
- Cross-Lingual Challenges: Asymmetries can also manifest across different language versions of Wikipedia, creating inconsistencies in how concepts are represented.
Key Results
- Across the four analyzed Wikipedia editions, only about half of the gender-related articles had symmetrical counterparts.
- The "W|G" (Women | Generic) tuple type was consistently the most frequent asymmetrical category, particularly in the "Sports" meta-category. For example, titles like "England women's national football team" often lacked a corresponding "England men's national football team" title, with the generic "England national football team" implicitly referring to the male team.
- The "Social" meta-category also showed significant asymmetry, often reflecting societal and biological differences or historical suppression that necessitated women fighting for rights.
- The study found that for German Wikipedia, gender marking is often achieved through suffixes (e.g., "-innen" for female) rather than explicit gendered words, posing a challenge for simple keyword-based filtering.
Limitations and Future Work
- Binary Gender Focus: The study acknowledges that its analysis is limited to binary gender values due to data scarcity for non-binary representations. This exclusion can indirectly cause bias against non-binary individuals.
- Title-Centric Approach: The methodology primarily focuses on article titles, potentially missing biases present in article content, hatnotes, or categories.
- Lexical and Semantic Asymmetries: While the method captures some imbalances, it may not fully account for all lexical (e.g., "La Liga" vs. "Primera División (women)") or semantic (e.g., "lady" vs. "woman") asymmetries.
- Future Work: The authors suggest extending the analysis to other Wikipedia components like hatnotes, interwiki links, and categories. They also propose developing debiasing strategies, such as adding explicit gender markers to generic titles or ensuring content aligns with titles to prevent mismatches. The debate on whether Wikipedia editors should reflect real-world imbalances or actively work to correct them is also highlighted as an area for further discussion.
Summary updated