Checking HateCheck: a cross-functional analysis of behaviour-aware learning for hate speech detection

Pedro Henrique Luz de Araujo, Benjamin RothACL 2022

TL;DR

This paper explores "behavioral-aware learning" for hate speech detection, investigating how models can be fine-tuned on specialized test suites (like HATECHECK) to improve their understanding of nuanced hate speech. While this approach shows promise in generalizing to new hate speech patterns, it also risks overfitting to the test suite's specific biases, potentially degrading performance on real-world data.

What Problem Does This Paper Address?

Evaluating Natural Language Processing (NLP) models, especially for sensitive tasks like hate speech detection, is challenging. Traditional methods rely on held-out datasets, which can overestimate performance and fail to reveal specific model weaknesses or biases. Behavioral testing, using carefully crafted input-output pairs that probe specific linguistic phenomena, offers a more diagnostic approach. However, the potential of using these behavioral tests during model training (behavioral-aware learning) to create more robust models has been underexplored. This paper addresses this gap by investigating how fine-tuning NLP models on a hate speech behavioral test suite (HATECHECK) impacts their performance, generalization capabilities, and potential for overfitting.

Methods

The core of this research involves fine-tuning pre-trained BERT models using different configurations of the HATECHECK dataset. They employ several "splitting techniques" for HATECHECK:

  • All: A standard random split of test cases for training and evaluation.
  • FuncOut: Holding out test cases from specific functionalities (e.g., implicit derogation, slur usage) during training.
  • IdentOut: Holding out test cases targeting specific identity groups (e.g., gender, race, religion).
  • ClassOut: Holding out entire functionality classes (e.g., derogation, slur usage).

They also explore different training configurations:

  • Task-only: Fine-tuning solely on standard hate speech detection datasets.
  • Test suite-only: Fine-tuning solely on HATECHECK.
  • Task and test suite: Sequentially fine-tuning first on task data, then on HATECHECK.

Performance is evaluated on both the original task datasets and various held-out subsets of HATECHECK to assess generalization.

Datasets

  • Task Data:
    • Davidson et al. (2017) dataset: Tweets annotated as hateful, offensive, or neither.
  • Founta et al. (2018) dataset: Tweets annotated as hateful, abusive, spam, or normal.
  • Both datasets were preprocessed to have binary "hateful" vs. "non-hateful" labels, matching HATECHECK.
  • Benchmark/Test Suite:
    • HATECHECK (Röttger et al., 2021): A suite of 3,728 hand-crafted and template-generated test cases designed to probe specific functionalities and aspects of hate speech detection. It covers 29 functionalities grouped into 11 classes and includes information on seven protected identity groups.

Key Insights and Ideas

  • Behavioral Tests as Training Data: The paper demonstrates that fine-tuning on behavioral test suites like HATECHECK can indeed improve a model's ability to handle specific, nuanced aspects of hate speech that might be missed by standard datasets.
  • Generalization vs. Overfitting: While models show improved performance on held-out functionalities and identity groups (suggesting generalization), this comes at the cost of decreased performance on standard, i.i.d. hate speech detection data. This highlights a critical trade-off: models learn the patterns of the test suite but may lose broader understanding.
  • Functionality vs. Identity Generalization: Models seem to generalize better across different identity groups than across different functionalities. This is attributed to how HATECHECK is constructed, where identity variations often use the same templates, while functionalities can be more distinct.
  • The Danger of Spurious Correlations: Fine-tuning on HATECHECK can lead models to overfit to specific linguistic patterns or "spurious correlations" present in the test suite. For example, certain spelling variations or phrasing, always associated with hateful examples in HATECHECK, can bias the model, even if the underlying text isn't inherently hateful.
  • Importance of Multi-faceted Evaluation: The study strongly advocates for evaluating models on both challenge datasets (like HATECHECK) and standard i.i.d. datasets. Passing behavioral tests doesn't guarantee real-world performance.

Key Results

  • Fine-tuning on HATECHECK significantly improved performance on held-out functionalities and identity groups compared to models trained only on task data.
  • However, performance on standard i.i.d. hate speech detection datasets (Davidson and Founta) decreased after fine-tuning on HATECHECK. Macro F1 scores dropped significantly.
  • The gap between "covered" and "held-out" performance was wider for functionalities than for identity groups.
  • Performance on held-out functionality classes (ClassOut) also decreased, suggesting poorer generalization to entirely new categories of hate speech.
  • Qualitative analysis revealed instances where models learned unintended associations, such as linking specific spelling variations to hateful content due to their prevalence in HATECHECK.

Limitations and Future Work

  • Overfitting to Test Suite Distribution: The primary limitation is that behavioral-aware learning, as implemented, leads to overfitting to the HATECHECK distribution, degrading performance on general hate speech detection tasks.
  • Limited Coverage of HATECHECK: HATECHECK covers only seven protected groups, leaving out many communities targeted by hate speech. This limits the scope of generalization analysis.
  • Future Work: The authors suggest exploring other types of behavioral tests, such as perturbation-based invariance tests, to mitigate overfitting to spurious correlations. They also emphasize the need for further research into how to effectively leverage behavioral tests for training without sacrificing performance on broader, real-world tasks.

Summary updated

BibTeX
@inproceedings{araujo2022checking,
  author = {Pedro Henrique Luz de Araujo and Benjamin Roth},
  title = {{Checking HateCheck: a cross-functional analysis of behaviour-aware learning for hate speech detection}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2022},
}
Figure