Accounting for Offensive Speech as a Practice of Resistance

Mark Diaz, Razvan Amironesei, Laura Weidinger, Iason GabrielNAACL 2022

TL;DR

This paper argues that current offensive speech detection models fail to capture the nuances of language used by marginalized communities as a form of resistance. They propose a relational understanding of offensiveness, focusing on the social context and intended outcomes of language, rather than just its denotative meaning, to build more responsible and effective NLP systems.

What Problem Does This Paper Address?

Current approaches to detecting offensive speech, such as hate speech and toxicity detection, primarily focus on identifying language that is denotatively offensive and likely to cause harm or disengagement. However, these methods often struggle with language that, while using terms considered offensive, is actually used beneficially within specific social contexts, particularly by marginalized communities. This paper highlights the gap in understanding how offensive language can serve as a practice of resistance, a tool for social bonding, and a mechanism for challenging oppressive norms. Without this nuanced understanding, NLP systems risk misclassifying and even censoring vital forms of expression used by these communities.

Methods

The authors propose a relational understanding of offensiveness, moving beyond a focus on the inherent properties of words or phrases. Instead, they emphasize the importance of:

  • Social Context: Understanding the network of social relations among producers, receivers, and perceivers of speech.
  • Intended Outcomes (Ends): Analyzing the purpose and impact of the language, whether it's to cause harm or to achieve prosocial goals like resistance and solidarity.
  • Mock Impoliteness: Examining how language that appears impolite on the surface can be used within a group to build social bonds and "inoculate" members against external hostility. This is exemplified through the use of offensive language by drag queens and draws an analogy to Cynic philosophy's use of offensive speech as a practice of resistance.

They advocate for a shift from simply classifying "toxicity" to understanding the sociological role of offensive language and its potential for beneficial impacts.

Datasets

Not explicitly discussed in the paper as datasets used for training or evaluation. The paper uses examples from online platforms like Twitter and references existing work that likely utilized such datasets.

Key Insights and Ideas

  • Offensive language as resistance: The paper argues that denotatively offensive language can be a powerful tool for marginalized communities to resist oppressive social norms and build solidarity.
  • The importance of "mock impoliteness": This concept, particularly within the queer community (e.g., drag queens), demonstrates how seemingly offensive language can be used playfully and beneficially within an in-group context.
  • "Context collapse" is a major challenge: Platforms like Twitter, where messages can be seen by a wide audience with varying levels of shared context, make it difficult to interpret the intended meaning of language.
  • Beyond denotation to relationality: Effective offensive speech detection requires moving beyond the literal meaning of words to understand the social dynamics, power structures, and intended effects of the language.
  • Cynic philosophy as a historical precedent: The paper draws parallels between modern practices of resistance through offensive language and the ancient Cynics' use of provocative speech to challenge societal norms.
  • NLP systems need to account for social identity and context: To accurately classify offensive speech, models must consider the social positions of speakers and audiences, as well as the historical and cultural context of language use.

Key Results

The paper does not present quantitative results from a specific model or experiment. Instead, it offers:

  • A conceptual framework for understanding offensiveness as a social practice of resistance.
  • An argument for the necessity of incorporating relational context and intended outcomes into offensive speech classification.
  • Illustrative examples from the queer community (drag queens) and Cynic philosophy to support their arguments.
  • Identification of practical challenges in data annotation and model design when trying to operationalize relational context.

Limitations and Future Work

  • Difficulty in operationalizing relational context: The paper acknowledges significant challenges in capturing and modeling the complex social and historical context required for accurate offensive speech classification.
  • Annotator bias and social identity: The authors note that annotators' own social identities and political views can influence their judgments, and that collecting this information raises ethical concerns.
  • Privacy concerns: Using user profile data or detailed social identity information for annotation and moderation raises privacy issues.
  • Monolithic group assumptions: The paper cautions against assuming that all members of a social group will interpret or use language in the same way.
  • Future Work: The authors suggest several directions:
  • Improving annotation task design by providing annotators with more contextual cues (e.g., dialect priming).
  • Exploring how annotators use contextual clues and sense-making practices.
  • Developing methods to capture and leverage user behavior and discourse styles.
  • Focusing on the "ends" of offensive language to better distinguish between harmful and beneficial uses.
  • Investigating the ethical implications of inferring social identity for annotation.

Summary updated

BibTeX
@inproceedings{diaz2022accounting,
  author = {Mark Diaz and Razvan Amironesei and Laura Weidinger and Iason Gabriel},
  title = {{Accounting for Offensive Speech as a Practice of Resistance}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2022},
}
Figure