Good Secretaries, Bad Truck Drivers? Occupational Gender Stereotypes in Sentiment Analysis

Jayadev Bhaskaran, Isha BhallamudiACL 2019

TL;DR

This paper investigates how sentiment analysis models perpetuate occupational gender stereotypes by introducing a new, gender-balanced dataset and evaluation methodology. They found that while simpler models reflect biases from their training data, advanced models like BERT can exhibit different, potentially pre-training-induced, biases, highlighting the complexity of debiasing.

What Problem Does This Paper Address?

The increasing application of sentiment analysis in critical areas like recruitment and employee evaluation raises concerns about the propagation of societal biases. Specifically, occupational gender stereotypes, deeply ingrained in our society, can be amplified by these models. Social Role Theory explains how observing gendered roles shapes our expectations, leading to rigid stereotypes about who is suited for which job. When these stereotypes are embedded in machine learning models, they risk reinforcing and perpetuating these biases, potentially leading to discriminatory outcomes. This paper aims to quantify and understand how sentiment analysis models, particularly those dealing with text, reflect and potentially amplify these occupational gender stereotypes.

Methods

The authors propose a novel methodology for evaluating gender stereotypes in sentiment analysis models. Their approach involves creating a gender-balanced dataset of sentences that assert a person's profession (e.g., "My brother is a nurse," "My sister is a nurse"). They then use this dataset to test three different sentiment analysis models:

  • Bag-of-words + Logistic Regression (Baseline): A traditional, simpler model.
  • BiLSTM: A recurrent neural network model that captures sequential information.
  • BERT: A state-of-the-art transformer-based model, pre-trained on a massive corpus.

The core evaluation involves measuring the difference in predicted positive sentiment probabilities between sentences with male and female nouns for the same profession. They employ statistical tests (paired t-tests with Bonferroni correction) to determine if these differences are statistically significant.

Datasets

  • Training Data: The models were trained on the Stanford Sentiment Treebank 2 (SST-2) dataset, a collection of movie review snippets with binary sentiment labels.
  • Evaluation Dataset: The authors introduce a new, custom-built dataset of 800 sentences. Each sentence follows the structure "noun is a/an profession," where the noun is either male or female, and the profession is one of 20 selected occupations. This dataset is balanced across genders, with 400 sentences per gender.

Key Insights and Ideas

  • Bias Propagation is Real: Sentiment analysis models, even those trained on seemingly neutral data, can pick up and propagate societal gender stereotypes.
  • Model Complexity Matters (Differently): Simpler models (BoW+LogReg, BiLSTM) tend to reflect biases present in their training data (SST-2), often showing higher sentiment for female-associated professions.
  • BERT's Pre-training Bias: The advanced BERT model, surprisingly, showed a tendency towards higher sentiment for male-associated professions. This suggests that biases might be introduced during the massive pre-training phase, rather than solely from the fine-tuning dataset.
  • Societal Stereotypes Reflected: The models' predictions for professions align with societal perceptions, with "secretary" often receiving high sentiment (though sometimes due to specific training data examples like the movie "Secretary") and "truck driver" consistently receiving low sentiment.
  • Semantic Pejoration: The study observed semantic pejoration in the "spinster" vs. "bachelor" pair, where the female term acquired a significantly more negative sentiment, mirroring societal language use.

Key Results

  • M.1 (BoW+LogReg) and M.2 (BiLSTM): Showed statistically significant differences, assigning higher positive sentiment to sentences with female nouns (F-M difference of +0.035 and +0.077, respectively). This aligns with biases found in the SST-2 training data.
  • M.3 (BERT): Showed a statistically significant difference in the opposite direction, assigning higher positive sentiment to sentences with male nouns (F-M difference of -0.040).
  • Occupational Sentiment: "Secretary" was identified as a high-sentiment profession by M.1 and M.2, while "truck driver" consistently received the lowest sentiment across all models. BERT rated "scientist" highest and "truck driver" lowest, reflecting a clear white-collar/blue-collar bias.
  • Spinster vs. Bachelor: The pair "spinster" and "bachelor" showed the largest sentiment difference, with "spinster" receiving a significantly lower predicted sentiment (-0.404).

Limitations and Future Work

  • Training Data Influence: The authors acknowledge that biases in the SST-2 training dataset significantly influence the simpler models. The influence of BERT's massive pre-training corpus on its biases is a key area for future investigation.
  • Dataset Specificity: While the new dataset is designed to be general, it focuses on specific sentence structures. Future work could explore more diverse sentence constructions and professions.
  • Debiasing Strategies: The paper doesn't propose debiasing methods but highlights the need for them. Future work could explore how to mitigate these identified biases, potentially by analyzing pre-training corpora or developing more robust fine-tuning techniques.
  • Downstream Applications: The authors suggest investigating occupational gender stereotypes in real-world downstream applications like automated resume screening, which is a critical area for bias mitigation.

Summary updated

BibTeX
@inproceedings{bhaskaran2019good,
  author = {Jayadev Bhaskaran and Isha Bhallamudi},
  title = {{Good Secretaries, Bad Truck Drivers? Occupational Gender Stereotypes in Sentiment Analysis}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2019},
}
Figure