Towards Legally Enforceable Hate Speech Detection for Public Forums

Chu Fei Luo, Rohan Bhambhoria, Xiaodan Zhu, Samuel DahanarXiv 2023 · DOI · arXiv

TL;DR

This paper introduces a novel task for hate speech detection that grounds models in legally enforceable definitions, moving beyond subjective interpretations. They release a new dataset annotated by legal experts and demonstrate that while large language models (LLMs) show promise, their reasoning capabilities still lag behind human experts.

What Problem Does This Paper Address?

Hate speech is a pervasive problem online, but detecting it in a way that aligns with legal frameworks is challenging. Existing hate speech detection models often rely on broad, subjective definitions that don't necessarily translate to legally enforceable standards. This inconsistency makes it difficult to use these models for actual legal enforcement or to ensure they are aligned with regulatory goals. The paper aims to bridge this gap by creating a system that can identify hate speech based on specific, legally defined criteria.

Methods

The core of this work is a new task formulation that grounds hate speech detection in eleven legally enforceable definitions. These definitions are drawn from criminal law, human rights codes, and hateful conduct policies from public forums. The researchers developed a novel dataset annotated by legal experts for violations of these definitions. They then experimented with using large language models (LLMs) for this task, employing zero-shot and few-shot prompting techniques. To improve performance, they also explored parameter-efficient fine-tuning and self-training methods. A key aspect of their methodology is the analysis of LLM reasoning by examining their explanations for classification decisions.

Datasets

The paper introduces a new, gold-label dataset annotated by ten volunteer legal experts. This dataset focuses on violations of eleven specific legal definitions of hate speech. They also created a "silver" dataset with automatically generated labels and a challenge dataset for unsupervised methods. The dataset was augmented with expert-edited samples and automatically generated data to increase its size and diversity.

Key Insights and Ideas

  • Legal Grounding is Crucial: Simply detecting "hate speech" isn't enough; it needs to be aligned with legally enforceable definitions to be useful for regulation and enforcement.
  • Subjectivity Persists: Even with legal definitions, there's still room for interpretation, leading to the "Unclear" label and highlighting the complexity of the task.
  • LLMs Struggle with Legal Reasoning: While LLMs can achieve decent classification performance, their explanations often lack the nuance and accuracy of legal experts, sometimes even hallucinating or repeating prompts.
  • Smaller Models Can Be Competitive: Some smaller LLMs, like Falcon-7b, showed competitive performance, suggesting that sheer parameter count isn't the only factor for success.
  • Fine-tuning Challenges: Fine-tuning encoder-only models like RoBERTa-Large on this task proved difficult, possibly due to the rarity of positive examples per definition, leading to model collapse.

Key Results

  • GPT-4 leads in coarse-grained classification: Achieving a Macro-f1 score of 92.5% with a very low hallucination rate (0.04%).
  • GPT-3.5 models perform comparably: GPT-3.5 achieved a Macro-f1 of 43.3% and GPT-3.5-16k achieved 43.4%, indicating that longer context windows didn't significantly improve performance in this specific task.
  • Smaller models show promise: Falcon-7b achieved a Macro-f1 of 17.4%, outperforming other smaller models and random guessing.
  • Reasoning degrades performance: When prompted to explain their decisions, LLMs generally saw a decrease in classification performance, with WizardLM notably repeating prompts.
  • Inter-annotator agreement is high: Krippendorff's alpha of 0.623 in the initial round indicates good consistency among legal experts.

Limitations and Future Work

  • Interpretation Subjectivity: Despite using legal definitions, the application of these definitions to specific text remains subject to interpretation, and the authors acknowledge their definition set might be culturally monotonous.
  • Out-of-Distribution Data: The authors note their definition of hate speech is still incompatible with other works, particularly those trained on different datasets or with different definitions.
  • LLM Limitations: Hallucinations, lack of interpretability, and the potential for models to generate repetitive or nonsensical explanations remain significant challenges.
  • Data Scarcity for Fine-tuning: The rarity of positive examples per definition makes fine-tuning smaller models difficult, leading to performance collapse.
  • Future Work: The authors suggest further investigation into LLM reasoning, exploring more robust fine-tuning strategies, and expanding the dataset to include definitions from other jurisdictions (e.g., the EU). They also highlight the need to address the potential for misuse of such tools and the importance of grounding them in legal expertise.

Summary updated

BibTeX
@misc{luo2023towards,
  author = {Chu Fei Luo and Rohan Bhambhoria and Xiaodan Zhu and Samuel Dahan},
  title = {{Towards Legally Enforceable Hate Speech Detection for Public Forums}},
  year = {2023},
  eprint = {2305.13677},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2305.13677},
}
Figure