Introduction
Popular metrics used to measure gender bias in artificial intelligence are fundamentally "polluted" by word frequency. In recent years, researchers have increasingly used word embeddings to audit societal biases in large text corpora. By measuring the geometric distance between a target word (e.g., "doctor") and gendered attribute words (e.g., "man" vs. "woman"), stereotypes can theoretically be quantified.
However, word embeddings encode more than just semantic meaning; they also capture word frequency. This study investigates whether current bias metrics are truly measuring societal stereotypes or merely reflecting the frequency of words in training data. If a metric is sensitive to frequency, it may falsely report bias simply because a word is common, leading to inaccurate conclusions about linguistic models.
Methodology
The researchers compared two primary methodologies for measuring bias:
- Word Embedding-based ($Bias_{WE}$): This calculates the difference in mean cosine similarity between a target word's vector and two sets of gendered attribute vectors (e.g., "he/him/man" vs. "she/her/woman"). Tests were conducted using GloVe and Word2Vec (SGNS).
- PMI-based ($Bias_{PMI}$): A statistical approach examining the probability of word co-occurrence within a text window, normalized by individual word frequencies.
To isolate the effect of frequency, a controlled experiment was conducted using a shuffled corpus. By taking the English Wikipedia and randomly relocating every word, all semantic context was destroyed while maintaining identical word frequencies. In this "gibberish" version of the text, a robust bias metric should return a score of zero, as no meaningful associations remain.
Datasets
- Pretrained Embeddings: GloVe (trained on Wikipedia 2014 + Gigaword 5) and Word2Vec/SGNS (trained on Google News).
- English Wikipedia (April 2021): A dump containing approximately 1.7 billion tokens used to train embeddings from scratch and create shuffled control datasets.
Key Insights and Results
The "Spurious Bias" Discovery
In the shuffled corpus, embedding-based metrics still detected significant gender bias despite the lack of semantic relationships. This demonstrates that the detected bias originates from the mathematical properties of the embedding algorithms rather than the text's meaning.
Frequency as a Proxy for Gender
In large corpora like Wikipedia, male-gendered words (e.g., "he", "him") appear significantly more often than female-gendered words (e.g., "she", "her"). Because algorithms like GloVe are sensitive to frequency, they tend to "pull" high-frequency target words toward the more frequent "male" cluster in vector space.
- GloVe exhibits a monotonic relationship: higher frequency words (lower rank) appear more "male," while lower frequency words (higher rank) appear more "female."
- SGNS (Word2Vec) is generally more stable but shows a sharp shift toward "male" bias for the top 100 most frequent words.
PMI as a Robust Alternative
Unlike embedding-based metrics, the PMI-based approach showed almost no dependence on frequency in shuffled experiments. It correctly identified that in random text, no meaningful bias exists.
In shuffled text, $Bias_{WE}$ for GloVe showed large Cohen’s $d$ effect sizes (exceeding 1.0) for high-frequency words, falsely indicating strong male bias. Across all experiments, a baseline skew toward male bias was observed, likely because male pronouns appear 3-4 times more frequently than female pronouns in the Wikipedia corpus.
Conclusion and Future Work
This study highlights a systemic issue in how word embeddings are constructed and audited. The authors urge the research community to move toward more robust metrics like PMI or to develop "frequency-agnostic" embedding techniques to ensure we are measuring actual human bias rather than statistical noise.
Limitations
- Binary Gender Constraint: The study treats gender as a binary (male/female), which does not capture the full spectrum of gender identity.
- Linguistic Scope: The analysis was restricted to English; languages with grammatical gender may exhibit different frequency-related artifacts.
- Hyperparameter Sensitivity: Future work should investigate if specific training settings (e.g., window size, vector dimensions) can mitigate frequency dependence.
Summary updated