TL;DR
This paper leverages Natural Language Processing (NLP) techniques, specifically word embeddings and sentiment analysis, to quantify gender bias and sentiment towards political leaders in Kenyan news over two decades. The findings reveal a complex picture: while media bias in leadership terms shows a trend towards equality, women leaders are increasingly associated with negative sentiment, suggesting a potential backlash to their growing representation.
What Problem Does This Paper Address?
Despite legislative efforts in Kenya to promote gender equality in political representation (e.g., the 2010 constitutional amendment aiming for no more than two-thirds of the same gender in elected bodies), women remain underrepresented in parliament. Understanding the socio-cultural barriers to this equality is crucial. Traditional methods for analyzing media bias are often manual and limited in scope. This paper addresses the need for a more efficient and scalable approach to quantify gender bias and sentiment in media coverage of political leaders, specifically within the Kenyan context, to better understand how media narratives might influence or reflect societal attitudes towards gender equality in leadership.
Methods
The researchers employed two primary NLP techniques:
- Word Embeddings (GloVe and word2vec): They trained word embedding models on a large corpus of text from Kenya's Daily Nation newspaper (1998-2019). Word embeddings represent words as vectors in a multi-dimensional space, where proximity indicates semantic similarity. By analyzing the relationships between gendered words (e.g., "woman," "man") and leadership-related words (e.g., "president," "leader"), they could quantify gender bias. They specifically measured the association between gender words and neutral leader words, and also between leader names and leader qualities.
- Sentiment Analysis: They used a pre-trained sentiment analysis model (AllenNLP) to assign a sentiment score (ranging from 0, very negative, to 4, very positive) to sentences. This was applied to sentences containing the names of male and female political leaders, as well as sentences immediately preceding and following them. This allowed them to compare the sentiment expressed towards male and female leaders over time.
A novel aspect of their approach is the creation of embeddings from a specific Kenyan news source, rather than relying solely on large, pre-trained, often Western-centric embeddings. They also combined insights from both word embedding bias and sentiment analysis for a more comprehensive understanding.
Datasets
- Kenya's Daily Nation Newspaper Archive (1998-2019): This formed the primary dataset for training the word embeddings and conducting the sentiment analysis. Over 500,000 articles were scraped.
- Inter-Parliamentary Union (IPU) Data: Publicly available data on the percentage of men and women in the Kenyan Parliament was used to correlate with temporal trends in media bias.
- Afrobarometer Survey Data: This dataset provided insights into public attitudes towards gender and leadership in Kenya, used for comparison with media trends.
The paper did not introduce a new benchmark dataset but rather utilized existing data sources to analyze trends within a specific media outlet.
Key Insights and Ideas
- Media Bias Mirrors Political Reality (to an extent): The study found a significant correlation between the temporal trends in word embedding bias (how leadership terms are associated with gender) and the actual participation of men and women in Kenyan parliamentary seats. This suggests that the media's portrayal, at least in terms of word associations, has somewhat kept pace with changes in political representation.
- Persistent Gendered Associations: Despite improvements, women leaders continue to be more strongly associated with "domestic terms" (e.g., homemaker, childcare), while men are more associated with "influence terms" (e.g., power, authority). This holds true for both general gender words and specific leader names.
- The "Backlash" Hypothesis: While gender bias in leadership terms shows a trend towards equality, sentiment analysis reveals a concerning increase in negative sentiment surrounding female leaders over time, alongside an increase in positive sentiment for male leaders. This suggests a potential media backlash or increased scrutiny as women gain more political prominence.
- NLP as a Powerful Tool for Social Science: The paper strongly advocates for NLP as an efficient and scalable method to quantify subtle, long-term trends in media bias and sentiment, offering insights previously difficult to obtain with traditional social science methods.
Key Results
- Word Embedding Correlation: GloVe embeddings showed a strong correlation (r=0.8936, p=0.0067) between leadership word bias and parliamentary seat trends, with word2vec showing a similar correlation (r=0.844, p=0.0169).
- Mention Disparity: Male words were mentioned 1.84 million more times than female words in the Daily Nation between 1998 and 2019.
- Sentiment Shift: A significant increase in relative negative sentiment associated with female leaders (p=0.0152) and a significant increase in positive sentiment associated with male leaders over time (p=0.0216) were observed.
- Leader Qualities: Adjectives related to "good leader" were increasingly associated with female leaders over time, while terms like "domestic" and "defamation" showed complex trends, sometimes becoming more associated with women.
Limitations and Future Work
- Single News Source: The analysis is based on a single newspaper (Daily Nation), which may not fully represent the views of the entire Kenyan population or other media outlets. Future work could involve analyzing multiple news sources, including those in local languages, and social media.
- Gender Binary: The study acknowledges that its analysis is limited to a gender binary, and future research could explore gender as a spectrum.
- Sentiment Model Limitations: The sentiment analysis model was trained on English text and may not perfectly capture nuances of Kenyan English. While a RoBERTa model was used for further analysis, its binary classification setting reduced precision.
- Wordlist Sensitivity: The findings are dependent on the chosen wordlists, and the authors acknowledge that these lists might not be exhaustive or perfectly representative.
- Linear Trends: The study primarily used linear regression, and the actual relationships might be non-linear.
- Future Directions: The authors suggest extending NLP analysis to social media, using newer contextual language models (like BERT), and analyzing media in local languages. They also propose using NLP as a monitoring tool for NGOs working on gender equality.
Summary updated