It's a Man's Wikipedia? Assessing Gender Inequality in an Online Encyclopedia

Claudia Wagner, David García, Mohsen Jadidi, Markus Strohmaier

TL;DR

While Wikipedia has made strides in representing notable women, a deep dive into its data reveals persistent systemic biases. This study shows that although women are covered and featured at rates comparable to men, they are portrayed through a different lexical lens—focusing disproportionately on family and relationships—and occupy more peripheral positions in the encyclopedia's internal link network.

What Problem Does This Paper Address?

Wikipedia is the world's most influential source of "neutral" information, yet its editor base is predominantly male. Researchers have long worried that this demographic imbalance creates a "gender gap" in content.

The authors argue that assessing this gap requires looking beyond simple headcounts. They tackle the challenge of defining and measuring gender inequality across four distinct dimensions:

  1. Coverage Bias: Are notable women less likely to have articles than notable men?
  2. Structural Bias: How are articles about men and women linked? Does one gender appear more "central" or important in the network?
  3. Lexical Bias: Does the language used to describe women differ from that used for men (e.g., focusing on domestic life vs. professional achievements)?
  4. Visibility Bias: Are articles about women less likely to be featured on the Wikipedia homepage?

Methods

The researchers employed a multi-dimensional computational approach across six language editions of Wikipedia (English, German, Spanish, French, Italian, and Russian).

  • Network Analysis: To measure structural bias, they used log-likelihood ratios to compare the probability of links between genders against null models (randomized networks). They also applied k-core decomposition and in-degree centrality to determine if women are relegated to the "periphery" of the knowledge graph.
  • Natural Language Processing (NLP): For lexical bias, they used an open-vocabulary approach. They trained a Naive Bayes classifier on word stems (using tf-idf scores) to identify which words most effectively distinguish the gender of an article's subject.
  • Statistical Testing: They used Chi-Square tests for visibility and Wilcoxon/Kolmogorov-Smirnov tests to compare distributions of article lengths and centrality.

Datasets

Because Wikipedia itself is the subject of study, the authors needed "ground truth" lists of notable people to check for coverage. They used three distinct reference datasets to mitigate individual dataset bias:

  • Freebase: A large-scale collection of ~120,000 notable people.
  • Pantheon: ~11,000 biographies with a presence in more than 25 languages.
  • Human Accomplishment (HA): ~4,000 eminent individuals in arts and sciences prior to 1950.

Key Insights and Ideas

  • The "Null Gender" Phenomenon: Articles about men rarely use words like "man" or "gentleman," whereas articles about women frequently use "woman," "female," or "lady." This suggests that in the Wikipedia ecosystem, "male" is often treated as the default human state, while "female" is a specified sub-category.
  • The Smurfette Principle: In the link network, women are often positioned in the periphery. While women's articles frequently link to men (often husbands or male colleagues), the reverse is less common. This creates a structural asymmetry where men appear more "essential" to the network.
  • The Finkbeiner Test Failure: The study confirms that Wikipedia often fails the "Finkbeiner Test" (a checklist to avoid sexism in biographies). Articles about women are significantly more likely to discuss romantic relationships, marriage, and family than articles about men of similar stature.

Key Results

  • Coverage & Visibility (The Good News): Surprisingly, women are slightly over-represented on Wikipedia relative to the reference datasets. Furthermore, there was no significant evidence of male bias in the selection of "Featured Articles" on the English Wikipedia homepage.
  • Structural Centrality: In English, Russian, and German editions, men are significantly more central in the link network. Women are more "assortative"—they tend to link to other women more than expected—but they still link to men more than men link to them.
  • Lexical Disparity: Between 23% and 32% of the most discriminative words for women fall into the categories of "Family," "Relationship," or "Gender." For men, these categories account for only 0% to 4% of their top discriminative words. Instead, men are defined by their fields (e.g., "baseball," "infantry").

Limitations and Future Work

The authors acknowledge that their "ground truth" reference datasets (like Freebase or HA) may themselves contain historical biases. If the real world has historically ignored notable women, Wikipedia's "equal" coverage might just be a reflection of an already biased history.

Future work could investigate whether these biases stem from the software design of Wikipedia, the specific behavior of male editors, or if Wikipedia is simply mirroring the lexical biases found in broader media. The authors also suggest that search and recommendation algorithms need to be audited to ensure they don't amplify the "peripheral" status of women's articles.

Summary updated

Abstract

Wikipedia is a community-created encyclopedia that contains information about notable people from different countries, epochs and disciplines and aims to document the world's knowledge from a neutral point of view. However, the narrow diversity of the Wikipedia editor community has the potential to introduce systemic biases such as gender biases into the content of Wikipedia. In this paper we aim to tackle a sub problem of this larger challenge by presenting and applying a computational method for assessing gender bias on Wikipedia along multiple dimensions. We find that while women on Wikipedia are covered and featured well in many Wikipedia language editions, the way women are portrayed starkly differs from the way men are portrayed. We hope our work contributes to increasing awareness about gender biases online, and in particular to raising attention to the different levels in which gender biases can manifest themselves on the web.
BibTeX
@inproceedings{wagner2015it,
  author = {Claudia Wagner and David García and Mohsen Jadidi and Markus Strohmaier},
  title = {{It's a Man's Wikipedia? Assessing Gender Inequality in an Online Encyclopedia}},
  booktitle = {ICWSM},
  year = {2015},
  url = {https://openreview.net/forum?id=DbCETfiPaQ},
}

Loading related content...

Figure