Analysis of Annotator Demographics in Sexism Detection

Narjes Tahaei, Sabine BerglerACL 2024

TL;DR

This paper investigates how annotator demographics (age, gender, ethnicity, education) influence the detection of sexism in social media. By incorporating these demographic features directly into BERT-based models, the authors demonstrate a significant improvement in both sexism classification and the identification of author intent, suggesting that accounting for annotator diversity leads to more robust and less biased models.

What Problem Does This Paper Address?

Identifying sexism in online content is a complex task because what one person considers sexist, another might not. This subjectivity leads to challenges in creating reliable datasets for training machine learning models. Traditionally, these models rely on a "majority vote" to determine the ground truth, which can overlook minority opinions and potentially embed biases from the annotator pool. The EXIST dataset, used in this study, aims to address this by collecting detailed demographic information from its annotators. However, the impact of these demographics on the labeling process and subsequent model performance had not been explored. This paper fills that gap by investigating how annotator demographics can be leveraged to improve sexism detection models.

Methods

The core of this research lies in exploring different ways to integrate annotator demographic information into a BERT-based model for sexism detection. The authors experimented with several "encodings" or methods of feeding this information to the model:

  • Encodingbase: This is the baseline, using only the tweet text as input to BERT.
  • Encodingfeatures: Demographic features are added as additional inputs directly to the classifier, separate from the BERT model.
  • Encodingannotations: This is the most promising approach. Here, each tweet is replicated six times (once for each annotator), and the annotator's demographic features are concatenated to the end of the tweet text. This allows the model to learn from the text in conjunction with the annotator's background.
  • Encodingannotations,sep: Similar to Encodingannotations, but demographic features are separated by a special [SEP] token.
  • Encodingannotations + labels: This further enhances Encodingannotations by also including the annotator's specific label for the tweet (e.g., "sexist," "direct") as a token.

The authors fine-tuned pre-trained models like RoBERTa and mBERT on the EXIST dataset, comparing performance across these different demographic feature integration strategies.

Datasets

The primary dataset used is the EXIST dataset (Plaza et al., 2023). This dataset contains Spanish and English tweets annotated for sexism. Crucially, it includes demographic information for each of its annotators, such as gender, age, level of study, and ethnicity. The dataset consists of 6,920 tweets annotated by 725 distinct annotators from 45 different countries. The paper uses the development set for evaluation due to the unavailability of public test data from the organizers.

Key Insights and Ideas

  • Demographics Matter: Annotator demographics are not just noise; they significantly influence how sexism is perceived and labeled. Ignoring them means potentially missing crucial nuances in the data.
  • Direct Integration is Key: Simply adding demographic features as separate inputs to a classifier isn't as effective as integrating them directly into the language model's input, especially when treated as tokens.
  • Tokenizing Demographics: Representing demographic features as tokens (similar to words) within the BERT input, rather than just numerical scalars or symbols, unlocks better performance. This suggests that BERT can learn meaningful representations from these demographic "words."
  • Annotator Labels as Features: Including the annotator's actual label for a given tweet, alongside their demographic information, provides a slight but noticeable performance boost. This implies that the model can learn from both the annotator's background and their specific judgment.
  • Bias is Multifaceted: The study highlights how factors like ethnicity, country of origin, and language spoken by annotators can lead to different rates of identifying sexism, underscoring the complexity of bias in annotation.

Key Results

The paper reports significant improvements in performance when incorporating annotator demographics.

  • Task 1 (Sexism Detection): The best performing model, using the Encodingannotations approach with demographic features as tokens (similar to Examples 4-6 in the paper), achieved a 5% performance increase compared to the baseline (Encodingbase).
  • Task 2 (Author Intent Classification): This task saw an even larger improvement, with a 7% increase in performance using the same best-performing demographic encoding.
  • Further Boost: Including the annotator's specific labels as tokens (Encodingannotations + labels) led to an additional 1% improvement for Task 1 and a 4% improvement for Task 2.
  • Qualitative Findings: The analysis of annotator distributions revealed about a 10% difference between demographic groups for most features. For instance, Spanish-speaking annotators labeled tweets as sexist at a higher rate (49%) than English-speaking annotators (41%).

Limitations and Future Work

The authors acknowledge that while incorporating demographic features can improve model performance and potentially lead to less biased datasets, it also carries the risk of introducing new biases.

Future work suggested includes:

  • Further Exploration of Demographic Effects: Investigating the precise mechanisms by which demographic features influence trained models.
  • Assessing Disagreement: Developing methodologies to better assess the degree of disagreement among annotators, building on existing work.
  • Addressing Potential Biases: Aiming for more robust models by better understanding and mitigating potential biases introduced by demographic data.

Summary updated

BibTeX
@inproceedings{tahaei2024analysis,
  author = {Narjes Tahaei and Sabine Bergler},
  title = {{Analysis of Annotator Demographics in Sexism Detection}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2024},
}
Figure