TL;DR
This paper investigates how individual worker characteristics and the structure of crowdsourcing platforms influence the subjective task of labeling misogynistic hate speech on Twitter. They found that annotator political leanings, moral integrity, and sexist attitudes significantly impact their labeling accuracy, and that the platform's structure and task instructions play a crucial role in shaping these decisions.
What Problem Does This Paper Address?
Automated content moderation systems, especially for nuanced and subjective content like hate speech, rely heavily on large, accurately labeled datasets. However, the process of labeling hate speech is inherently subjective, making it susceptible to biases introduced by the annotators themselves. These biases can stem from their personal beliefs, demographics, and even the way the task is presented. This paper specifically tackles the challenge of understanding and quantifying these biases in the context of misogynistic hate speech on Twitter, aiming to improve the quality and reliability of data used for training moderation models.
Methods
The researchers conducted two studies on Amazon Mechanical Turk (MTurk).
- Study 1 (Quantitative): 109 crowd workers were tasked with labeling tweets for hate speech. This involved two phases: first, a binary classification (hate speech vs. not hate speech), and second, a fine-grained classification into five specific categories of misogynistic hate speech. Following the annotation, workers completed questionnaires measuring their political inclination, moral identity, personality traits (Big Five), and ambivalent sexism.
- Study 2 (Qualitative): Nine crowd workers from Study 1 participated in semi-structured interviews. They were asked to share their screens and talk through their decision-making process while annotating, providing insights into their sense-making strategies, the influence of task instructions, and their personal motivations.
Datasets
The study utilized a custom dataset of 140 tweets, specifically curated to include misogynistic or sexist hate speech. This dataset was derived from the tweets of seven public figures identified as being susceptible to such content. Expert annotations were used as a gold standard for evaluation. The dataset includes 90 non-hateful tweets and 50 hateful tweets, further categorized into five sub-types of hate speech: Threats and Violent Abuse (TVA), Sexualised Abuse (SA), Other Objectifying Speech (OOS), Other Contemptuous Speech (OCS), and Other Hate Speech (OHS).
Key Insights and Ideas
- Political Leanings Matter: Annotator political inclination was a significant predictor of both accuracy and the tendency to label tweets as hate speech. Left-leaning annotators, in particular, showed higher accuracy and a lower rate of labeling tweets as hate speech.
- Moral Compass and Sexism Influence Judgments: A strong negative correlation was found between moral integrity and annotation accuracy, suggesting that workers with a higher desire to align actions with moral principles might struggle with the task's inherent subjectivity or conflicting instructions. Conversely, benevolent sexism was negatively correlated with accuracy, while hostile sexism showed a positive correlation with the percentage of hate speech annotations.
- Platform Structures Shape Decisions: The qualitative interviews revealed that financial incentives, platform approval ratings, and the fear of task rejection heavily influenced workers to prioritize task instructions over their personal values. This highlights how the "precarious work" nature of crowdsourcing can override individual judgment.
- Subjectivity is a Negotiation: Workers don't just apply pre-defined rules; they actively negotiate the task based on platform structures, instructions, their own motivations, and external context. This complex interplay shapes their annotations.
- Fine-grained Labeling is Challenging: While workers performed better in binary classification, accurately categorizing tweets into specific types of misogynistic hate speech proved difficult, with low agreement even among those who correctly identified hate speech.
Key Results
- Inter-Annotator Agreement: Krippendorff's alpha scores were low (0.149 in Phase 1, 0.161 in Phase 2 for binary classification, and 0.086 for fine-grained classification), indicating only slight agreement among workers, which is noted as common in hate speech annotation tasks.
- Political Impact: Left-leaning workers had significantly higher accuracy (Phase 1: M=70.3 vs. M=57.5/62.6; Phase 2: M=68.7 vs. M=52.5/53.6) and a lower percentage of hate speech annotations (Phase 1: M=28.9 vs. M=41.5; Phase 2: M=45.4 vs. M=62.1/68.1) compared to center and right-leaning workers.
- Gender Impact: No significant differences in accuracy were found based on gender. However, women tended to mark more content as hate speech than men in Phase 1.
- Qualitative Findings: Participants frequently discussed the importance of free speech, with definitions of hate speech being geographically and culturally bound. They also described using strategies like looking for keywords, re-reading context, and consulting external resources (e.g., Urban Dictionary) to fill in gaps in instructions.
Limitations and Future Work
- Dataset Size and Scope: The dataset, while curated, is relatively small (140 tweets) and derived from only seven public figures, limiting generalizability.
- Qualitative Sample Size: The qualitative study involved only nine participants, which is a small sample for in-depth thematic analysis.
- Self-Reported Data: The study acknowledges the inherent challenge of validating self-reported demographics and questionnaire outcomes in crowdsourcing settings.
- Future Work: The authors suggest that their findings can inform better pre-selection of workers or task assignment strategies. They also emphasize the need for informed pre-selection that considers a broader range of factors beyond demographics to avoid introducing new biases. They advocate for a more holistic accounting of influential factors in crowdsourcing, moving beyond just individual worker characteristics to include the impact of platform structures and task design.
Summary updated