Who Speaks Matters: Analysing the Influence of the Speaker’s Linguistic Identity on Hate Classification

Ananya Malik, Kartik Sharma, Shaily Bhatt, Lynnette Hui Xian NgEMNLP 2025

TL;DR

This paper investigates how the linguistic identity of a speaker, whether explicitly stated or implicitly conveyed through dialect, affects the hate speech classification performance of Large Language Models (LLMs). The findings reveal that LLMs are surprisingly brittle, with implicit dialect markers causing more classification flips than explicit mentions, highlighting a critical need for caution when deploying these models in high-stakes moderation tasks.

What Problem Does This Paper Address?

Large Language Models (LLMs) are increasingly being adopted for content moderation, including hate speech detection, due to their scalability. However, LLMs are known to be brittle and can exhibit biases, particularly against marginalized communities and dialects. This poses a significant risk when deploying them in high-stakes scenarios like hate speech detection, where misclassifications can have serious consequences. While prior work has explored the impact of assigning cultural identities to models, this paper specifically addresses the gap in understanding how the linguistic identity of the speaker, conveyed through their dialect, influences hate speech classification. The authors aim to quantify this influence and understand its implications for the reliability of LLM-based hate speech detection systems.

Methods

The core of this research involves systematically injecting linguistic identity markers into input sentences and observing how LLM hate speech classifications change. Two types of markers were used:

  • Explicit Markers: A phrase directly mentioning the speaker's linguistic identity (e.g., "The Jamaican person said...").
  • Implicit Markers: Dialectal features representative of specific linguistic identities were incorporated into the sentences. This involved generating dialectal variations of hate speech examples using a few-shot Llama-3-70B model, carefully ensuring the original meaning and syntax were preserved while introducing colloquialisms, cultural themes, code-mixing, and region-specific spellings.

The researchers then measured the "flip rate" – the percentage of times the model's classification changed from the original, unmarked input when these identity markers were present. They evaluated four language models: a fine-tuned BERT model, Llama-3-8B, Llama-3-70B, and GPT-4o, using both zero-shot and in-context learning (ICL) prompting strategies.

Datasets

The study utilized two datasets:

  • MPBHSD: Derived from Twitter, 4Chan, and Reddit, containing 600 Hateful (H) and 2400 Non-Hateful (NH) sentences.
  • HateXplain: Derived from Twitter and Gab, comprising 3000 sentences (2094 Hateful, 906 Non-Hateful).

The dialectal variations for the implicit markers were generated using the Llama-3-70B model, with a human verification step to ensure the quality and validity of the generated data.

Key Insights and Ideas

  • Dialect Matters More Than Explicit Mention: Implicit dialectal markers caused significantly more classification flips than explicit mentions of identity. This suggests that LLMs are more sensitive to subtle linguistic cues than direct statements about identity.
  • LLMs are Brittle: The study demonstrates a notable brittleness in LLMs, where the introduction of seemingly irrelevant linguistic identity information can alter hate speech classifications.
  • Model Size and Recency Correlate with Robustness: Larger and more recent models (Llama-3-70B, GPT-4o) generally showed a smaller percentage of flips, indicating they are more robust to these identity manipulations compared to smaller models like Llama-3-8B.
  • ICL Offers More Stability: In-context learning (ICL) generally resulted in lower flip percentages compared to zero-shot learning, suggesting that providing examples can lead to more stable and consistent model outputs.
  • Identity-Specific Biases: The percentage of flips varied across different ethnic identities, indicating that LLMs may have learned biases that make them more susceptible to certain dialects or identities.

Key Results

  • Flip Percentages: Table 2 and Figure 2 show aggregate flip percentages across different dialects and models. For instance, implicit markers on the HateXplain dataset showed flips ranging from around 8% to over 40% depending on the model and dialect.
  • Model Performance: While models generally performed well on unmarked inputs (Table 1, up to 90% accuracy on MPBHSD), their performance degraded when identity markers were introduced.
  • False Negatives Increase: The study found that hateful (H) predictions were more likely to flip to non-hateful (NH) predictions across most models when identity markers were present. This suggests an increase in false negatives, meaning harmful content might go undetected.
  • Llama-3-8B's Instability: The Llama-3-8B model, particularly in its ICL variant, exhibited an unusually high percentage of flips (around 40% on MPBHSD), suggesting it remains unstable even with ICL prompting.

Limitations and Future Work

The authors acknowledge several limitations:

  • Limited Dialect Data: The study relied on sampled and verified data, but it may not cover all dialects or capture all nuances of code-mixed English. The synthetic generation of dialectal data, while necessary, might introduce its own biases. Future work could focus on more interpretable studies of these synthetic phrases.
  • Limited Models: Due to computational constraints, the study could not evaluate all models, particularly those advertised as "safer" (e.g., Claude). Preliminary experiments with Llama Guard were hindered by model refusals.
  • Limited Hate-Speech Datasets: The study was confined to dialect-mixed English datasets. Findings from multilingual datasets might differ.

Future work could explore more interpretable studies on the impact of specific dialectal features, expand the range of models and datasets, and investigate multilingual contexts. The authors also suggest further research into how LLMs handle sarcasm and negative framing, as observed with Llama-3-8B.

Summary updated

BibTeX
@inproceedings{malik2025who,
  author = {Ananya Malik and Kartik Sharma and Shaily Bhatt and Lynnette Hui Xian Ng},
  title = {{Who Speaks Matters: Analysing the Influence of the Speaker’s Linguistic Identity on Hate Classification}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
}
Figure