Reconsidering Annotator Disagreement about Racist Language: Noise or Signal?

Savannah Larimore, Ian Kennedy, Breon Haskett, Alina Arseniev-KoehlerNAACL 2021

TL;DR

This paper investigates why human annotators disagree on whether tweets contain racist language. They found that annotator race significantly influences how tweets are perceived, particularly for certain topics, suggesting that annotator disagreement is a valuable signal of differing interpretations rather than just noise. This has critical implications for building fair and accurate AI systems to detect hate speech.

What Problem Does This Paper Address?

Developing AI systems to detect hateful and racist language online is a crucial area of research, with potential applications ranging from content moderation to providing trigger warnings. However, a major hurdle is the low agreement among human annotators who label this data. This disagreement is often treated as "noise" to be smoothed over, and the resulting AI models may not accurately reflect the nuances of how racism is perceived. Furthermore, much of this work is disconnected from established social science theories of race and racism. This paper tackles this problem by exploring whether annotator disagreement is actually a meaningful "signal" that can be explained by factors like annotator race and the specific content of the tweets, drawing on social science perspectives.

Methods

The researchers employed a multi-faceted approach to analyze annotator disagreement. They used Structural Topic Modeling (STM) to identify latent topics within tweets, moving beyond simple keyword matching. They then built linear regression models to test their hypotheses. Specifically, they examined:

  • Annotator Race Model: How annotator race (White vs. non-White) alone predicts racial sentiment ratings.
  • Text Features Model: How keywords and latent topics influence sentiment ratings, controlling for annotator race.
  • Interaction Model: Crucially, this model tested how the interaction between annotator race and specific tweet topics affects sentiment ratings. This allowed them to see if certain topics are interpreted differently by annotators of different racial backgrounds.

They also included covariates like annotator gender, age, and tweet features (URLs, mentions, length) to control for other potential influences.

Datasets

The study combined data from two separate research projects, resulting in a dataset of 5188 unique tweets annotated by 291 human raters. The tweets were sourced from Twitter's Streaming API and the Digital Online Life and You Project (DOLLY), restricted to those sent from the contiguous United States and written in English. Keywords related to hate speech, "black girl magic," and general terms for Black Americans were used to sample relevant tweets. Annotators were recruited via Amazon Mechanical Turks and college students.

Key Insights and Ideas

  • Disagreement is a Signal, Not Just Noise: The core idea is that annotator disagreement isn't just random error; it reflects genuine differences in how people perceive racism, influenced by their own identities and experiences.
  • Race Matters in Perception: Annotator race is a significant factor in how racist language is interpreted, especially when certain topics are present in a tweet. This challenges the notion of a universal, objective interpretation of racist content.
  • Context is King (and Race-Dependent): The type of topic within a tweet, combined with the annotator's race, dramatically influences sentiment ratings. This highlights the need to move beyond simple "bag-of-words" approaches.
  • Social Science Theory is Crucial: The paper emphasizes that understanding racism requires drawing on established social science theories, not just computational methods.

Key Results

  • Overall Agreement is Low: Krippendorff's alpha for overall inter-rater reliability was 0.39, indicating low agreement. However, agreement was slightly higher among White raters (0.44) than non-White raters (0.34).
  • Main Effect of Race is Small: While White raters, on average, rated tweets as having slightly higher racial sentiment (Model 1), this effect was small and statistically significant but not practically substantial on its own.
  • Text Features are Highly Influential: All identified topics and keywords significantly influenced sentiment ratings (Model 2), showing that annotators are indeed responding to the content.
  • Interaction Effects are Strong: The most significant finding (Model 3) was that the interaction between annotator race and seven of the twelve identified topics significantly impacted sentiment ratings. This means that for specific topics, White and non-White raters interpreted tweets very differently. For example, tweets about "Police Brutality" were rated "moderately positive" by White raters but closer to "neutral" by non-White raters.

Limitations and Future Work

  • Generalizability of Annotators: The study acknowledges that MTurk workers and college students may not perfectly represent the general Twitter user population, with a potential overrepresentation of young, White, and well-educated individuals.
  • Focus on Anti-Black Racism: The study specifically focused on anti-Black racism, and findings may differ for other forms of racism.
  • Future Directions:
    • Purposeful Sampling: Researchers should actively seek out diverse annotator populations.
  • Report Demographics: Annotator demographics should be collected and reported to provide transparency.
  • Weighted Annotations: Consider weighting annotations based on how directly a group is impacted by the type of racism being analyzed (e.g., giving Black individuals' interpretations more weight for anti-Black racism).
  • Qualitative Rationale: Future work could delve deeper into why annotators make certain ratings, exploring their rationale.

Summary updated

BibTeX
@inproceedings{larimore2021reconsidering,
  author = {Savannah Larimore and Ian Kennedy and Breon Haskett and Alina Arseniev-Koehler},
  title = {{Reconsidering Annotator Disagreement about Racist Language: Noise or Signal?}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2021},
}
Figure