The Origin and Value of Disagreement Among Data Labelers: A Case Study of Individual Differences in Hate Speech Annotation

Yisi Sang, Jeffrey M. StantoniConference 2022 · OpenReview

TL;DR

This paper tackles the challenge of subjective data labeling, specifically in hate speech annotation, where annotator disagreement is common and often discarded. They developed a multidimensional scale to understand why annotators disagree, revealing that individual differences like age and personality significantly influence their judgments. This work offers a path to better understand and potentially leverage these "minority-vote" labels in content moderation.

What Problem Does This Paper Address?

In the world of AI, human-annotated data is crucial for training models. However, for tasks involving subjective interpretations, like identifying hate speech, getting consistent labels from different annotators is a major hurdle. The standard practice of using majority votes to resolve disagreements can mask valuable nuances. This paper argues that for subjective tasks, simply discarding dissenting opinions is a missed opportunity. They aim to understand the root causes of these disagreements, particularly focusing on how individual annotator characteristics contribute to varying interpretations of hate speech.

Methods

The researchers employed a mixed-method approach to develop their multidimensional scale. This involved:

  • Literature Review: Compiling 98 definitions of hate speech to identify key elements.
  • Expert Interviews: Conducting seven semi-structured interviews with hate speech researchers to gather insights into their evaluation processes.
  • Concept Mapping: Presenting themes from interviews to ten hate speech experts who then sorted them into categories, leading to an eight-facet taxonomy of hate speech evaluation.
  • Scale Development: Generating and refining items for the scale based on this taxonomy, focusing on constructs like "Violence Tendency," "Impact," "Insult to Human Dignity," "Slander," "Moral Violation," "Radicalization," "Showing Fear," "Target," and "Groundless Accusation."
  • Statistical Analysis: Using Exploratory Factor Analysis (EFA) to identify underlying factor structures and Confirmatory Factor Analysis (CFA) to validate the proposed five-factor model.
  • Multigroup Analysis: Employing Confirmatory Factor Analysis (CFA) across different groups (based on age and personality) to assess measurement invariance, i.e., whether the scale measures the same thing across these groups.

Datasets

The paper does not introduce a new benchmark dataset for hate speech annotation. Instead, it focuses on developing a scale to evaluate existing or newly collected hate speech instances. The data used for scale development and testing consisted of:

  • Hate Speech Examples: One hate speech example and one neutral example were used for initial survey fielding.
  • Annotator Data: Data was collected from 170 annotators who participated in a hate speech annotation task. Participants were recruited from the Prolific.co platform.

Key Insights and Ideas

  • Subjectivity is Inherent: The paper strongly suggests that subjectivity is not just a nuisance but a fundamental aspect of hate speech annotation. Different annotators bring their own "mental schemas" shaped by experience and personality, leading to varied interpretations.
  • Beyond Majority Vote: Disagreements aren't just noise; they can reveal valuable information about how different people perceive and interpret potentially harmful content. This suggests that minority opinions might hold important insights.
  • Age and Personality Matter: Individual differences, specifically age and personality traits (like Extraversion, Agreeableness, Conscientiousness, Neuroticism, and Openness to Experience), significantly influence how annotators evaluate hate speech. This means a "one-size-fits-all" approach to annotation training might be insufficient.
  • Multidimensional Nature of Hate Speech: The developed scale confirms that hate speech is not a monolithic concept but can be broken down into several distinct evaluative dimensions, such as violence tendency, impact, and the nature of the target.

Key Results

  • Five-Factor Model: Exploratory Factor Analysis (EFA) suggested a five-factor solution for evaluating hate speech, which was then validated through Confirmatory Factor Analysis (CFA). The final model used 13 items.
  • Measurement Invariance Issues: Multigroup analysis revealed that the scale does not exhibit measurement invariance across different age groups and personality profiles. This means that people of different ages and with different personality traits tend to interpret the scale items differently, leading to variations in their hate speech evaluations.
  • For example, strict invariance tests showed significant differences for age groups on "Violence Tendency" and "Groundless Accusation."
    • Similarly, personality traits like Conscientiousness and Neuroticism showed significant differences in their influence on evaluating "Violence Tendency" and "Showing Fear."
  • Annotator Differences: The study found that age and personality had a substantial influence on the dimensional labeling of hate speech, suggesting that achieving perfect annotation consistency among diverse labelers is challenging.

Limitations and Future Work

  • Limited Stimuli: The study acknowledges that only two stimuli were used to generate variance in item responses, which might have inflated inter-construct correlations. Future work should use a wider variety of speech content.
  • Scale Validation: The newly developed evaluative scales require further validation and testing on a broader range of speech content.
  • Partitioning Variance: Future research should aim to partition evaluator variance from other sources of variance more effectively.
  • Generalizability: The study used hate speech as an example, but the multidimensional scale approach is applicable to other subjective data labeling tasks (e.g., misinformation, toxicity). The authors plan to explore these in the future.
  • Implications for AI: The findings have implications for designing content moderation systems, suggesting that future systems might need to be adaptable to different demographic groups or community contexts, potentially by incorporating information about annotator variance.

Summary updated

Abstract

Human annotated data is the cornerstone of today’s artificial intelligence efforts, yet data labeling processes can be complicated and expensive, especially when human labelers disagree with each other. The current work practice is to use majority-voted labels to overrule the disagreement. However, in the subjective data labeling tasks such as hate speech annotation, disagreement among individual labelers can be difficult to resolve. In this paper, we explored why such disagreements occur using a mixed-method approach – including interviews with experts, concept mapping exercises, and self-reporting items – to develop a multidimensional scale for distilling the process of how annotators label a hate speech corpus. We tested this scale with 170 annotators in a hate speech annotation task. Results showed that our scale can reveal facets of individual differences among annotators (e.g., age, personality, etc.), and these facets’ relationships to an annotator’s final label decision of an instance. We suggest that this work contributes to the understanding of how humans annotate data. The proposed scale can potentially improve the value of the currently discarded minority-vote labels.
BibTeX
@inproceedings{sang2022origin,
  author = {Yisi Sang and Jeffrey M. Stanton},
  title = {{The Origin and Value of Disagreement Among Data Labelers: A Case Study of Individual Differences in Hate Speech Annotation}},
  booktitle = {iConference},
  year = {2022},
  url = {https://openreview.net/forum?id=whr2dtNOD6},
}
Figure