TL;DR
Deploying Large Language Models (LLMs) for automated content moderation seems efficient, but how robust are they to cultural and regional variations in speech? This paper demonstrates that injecting a speaker's linguistic identity—either explicitly by stating their nationality or implicitly through regional dialects—causes models to frequently flip their hate speech classifications. Alarmingly, implicit dialect markers significantly increase false negatives, allowing actual hate speech to slip through undetected.
What Problem Does This Paper Address?
Automated hate speech detection is a high-stakes task where errors can either silence marginalized communities (false positives) or allow toxic behavior to proliferate (false negatives). While LLMs perform well on standard benchmarks, they are heavily biased toward Standard American English.
The authors investigate a critical gap: How robust are these models when the inputs contain markers of the speaker's linguistic identity? If a user writes a comment using African-American English, British English, Indian English, Jamaican Patois, or Singaporean English (Singlish), does the model's classification change even if the core semantic meaning remains the same?
Methods
The researchers evaluated the robustness of four models: a fine-tuned HateXplain-BERT model, Llama-3-8B, Llama-3-70B, and GPT-4o. They tested two prompting strategies: zero-shot classification and in-context learning (ICL) with four examples.
To evaluate the impact of linguistic identity, they modified baseline sentences using two types of markers:
- Explicit Markers: They prepended a phrase indicating the speaker's background (e.g., The [ethnicity] person said, "[input]").
- Implicit Markers: They transformed the baseline text into specific regional dialects (e.g., adding colloquialisms, code-mixing, or region-specific spellings) using a few-shot Llama-3-70B model, followed by human verification to ensure semantic meaning was preserved.
The core metric tracked was the flip percentage—how often a model changed its prediction (from hateful to non-hateful, or vice versa) after an identity marker was introduced.
Datasets
The study utilized two public hate speech benchmarks:
- HateXplain: A dataset derived from Twitter and Gab. The authors sampled 3,000 sentences (2,094 Hateful, 906 Non-Hateful).
- MPBHSD (Multi-Platform Bias-Mitigated Hate Speech Dataset): Sourced from Twitter, 4chan, and Reddit, comprising 600 Hateful and 2,400 Non-Hateful sentences.
Key Insights and Ideas
- Implicit Dialects Break Models More Than Explicit Labels: Models are relatively stable when told "A Jamaican person said X." However, when the text is actually written in Jamaican dialect, the classification accuracy degrades sharply.
- The Asymmetry Danger (More False Negatives): When predictions flipped, they overwhelmingly flipped from Hateful to Non-Hateful (H $\rightarrow$ NH). This means that when hate speech is spoken in a non-standard dialect, LLMs fail to recognize it, creating a severe safety loophole.
- In-Context Learning (ICL) Acts as a Stabilizer: Providing few-shot examples in the prompt consistently lowered the flip percentage compared to zero-shot prompting, suggesting that examples help anchor the model's decision boundaries across different dialects.
- Model Size Correlates with Robustness: Larger, newer models like Llama-3-70B and GPT-4o exhibited significantly fewer flips than the smaller Llama-3-8B model, though they were still far from immune.
Key Results
Without any identity markers, the models achieved high baseline performance (up to 90% accuracy on MPBHSD and 80% on HateXplain). However, the introduction of implicit dialect markers caused significant performance degradation:
- HateXplain-BERT saw its predictions flip up to 43.4% of the time on the HateXplain dataset when implicit dialect features were introduced.
- Llama-3-8B (ICL) proved highly unstable on the MPBHSD dataset, with flip rates hovering around 40% across all tested dialects.
- Even GPT-4o (Zero-shot) experienced a 26.96% flip rate on African-American dialect inputs within the HateXplain dataset.
Statistical validations (t-tests and McNemar's tests) confirmed that the speaker's linguistic identity significantly alters classification outcomes ($p < 0.05$).
Limitations and Future Work
The authors note several limitations in their current work:
- Synthetic Dialect Data: Due to a lack of real-world, human-annotated hate speech datasets in multiple dialects, the implicit text had to be synthetically generated via an LLM. While human-verified, this approach may introduce synthetic artifacts.
- English-Centric Focus: The study is limited to English-based dialects and does not explore the complexities of fully multilingual hate speech detection.
- Model Constraints: Highly guarded commercial models like Claude were not evaluated due to resource constraints and safety filter refusals during testing.
Future research should focus on developing more interpretable evaluation frameworks to pinpoint exactly which dialectal phrases trigger these classification failures.
Summary updated