On the Effectiveness of Adversarial Robustness for Abuse Mitigation with Counterspeech

Yi-Ling Chung, Jonathan BrightNAACL 2024 · DOI

TL;DR

This paper investigates how well automated counterspeech systems can handle real-world online abuse, especially when faced with adversarial attacks designed to fool them. They introduce new datasets and find that while adversarial training can improve robustness, it doesn't always lead to better generalization, and authentic counterspeech differs significantly from synthetic examples.

What Problem Does This Paper Address?

Online abuse is a major societal issue, and counterspeech – responding to abuse with constructive dialogue – is a promising mitigation strategy. However, current automated counterspeech systems often rely on synthetic data, which may not reflect the complexities of real-world interactions. This gap raises questions about how robust these models are against adversarial attacks and how well they can generalize to unseen, user-generated abusive content. The paper aims to bridge this gap by exploring the effectiveness of adversarial robustness for abuse mitigation using authentic and synthetically generated counterspeech, particularly in the context of public figures.

Methods

The authors employ a multi-pronged approach:

  • Data Collection: They curate two novel datasets:
  • PLF Dataset: A dataset of 6,955 labeled tweets targeting footballers, capturing authentic abuse and replies.
  • DynaCounter Dataset: A dataset of 1,911 abuse-reply pairs covering nine minority identity groups, collected through an adversarial human-in-the-loop process over four rounds. This dataset is specifically designed to test model robustness against adversarial attacks.
  • Adversarial Attack and Testing: They use a "model-in-the-loop" approach where human annotators attempt to craft counterspeech that tricks a trained classifier. This process is repeated over several rounds, with the classifier being retrained on the newly collected adversarial data.
  • Counterspeech Generation: They evaluate two large language models (T5 and Flan-T5) on their ability to generate counterspeech for various abuse targets, assessing both automatic metrics and human evaluations.

Datasets

  • PLF Dataset: A newly curated dataset of 6,955 labeled tweets targeting footballers, focusing on authentic abuse and replies.
  • DynaCounter Dataset: A newly introduced dataset of 1,911 abuse-reply pairs covering nine minority identity groups, collected via an adversarial human-in-the-loop process. This dataset is designed to evaluate adversarial robustness.
  • Existing Datasets: The paper also utilizes existing counterspeech datasets for training classifiers, including CONAN, MTCONAN, KNNCONAN, Gab+Reddit, and TOXIGEN.

Key Insights and Ideas

  • Authentic vs. Synthetic Counterspeech: Authentic counterspeech, especially from users, tends to be informal, emotive, and direct, often incorporating slang. Expert-generated counterspeech, on the other hand, is more formal, evidence-based, and structured. This distinction is crucial for developing effective mitigation tools.
  • Adversarial Training Limitations: While adversarial training can improve model robustness, it doesn't always guarantee better generalization. The paper found that performance gains from adversarial data were not consistently observed across iterations, and sometimes even led to performance drops.
  • Difficulty of Certain Targets: Some abuse targets are inherently more challenging for counterspeech generation. For instance, both T5 and Flan-T5 struggled to produce satisfactory responses for abuse targeting footballers, and T5 had issues with topics related to Jews.
  • Human-in-the-Loop is Key: The adversarial data collection process, involving human annotators actively trying to fool models, is essential for uncovering model weaknesses and creating more robust systems.

Key Results

  • Dataset Statistics: The PLF dataset contains 6,956 pairs of abusive tweets and replies, with most replies disagreeing with the abuse (62%) and not supporting the targets (93%).
  • Adversarial Model Performance: Adversarial models (M1-M4) generally achieved higher macro F1 scores than the baseline on non-adversarial test sets, but performance gains were not always consistent across iterations.
  • DynaCounter Performance: The baseline model achieved an average F1 score of 0.46 on the DynaCounter test data, indicating that PLF data itself can be considered adversarial. Adversarial models showed lower performance on later rounds of DynaCounter data, suggesting the collected data became progressively more challenging.
  • Counterspeech Generation Usability: Language models (Flan-T5 and T5) generated usable counterspeech responses approximately 69-76% of the time. Flan-T5 was generally perceived as more usable and slightly more engaging, specific, and comprehensible than T5.
  • Model Error Rate: The model error rate in the adversarial attack process decreased over rounds, indicating that the models learned to resist some adversarial attempts. However, examples using emojis or hashtags had higher error rates.

Limitations and Future Work

  • Subjectivity of Labels: Annotators' perceptions of suitable counterspeech can vary, and labels are based on majority votes or expert judgment, which can introduce subjectivity.
  • Synthetic Training Data: The datasets used for training classification models are synthetically created and not fully validated on actual real-world abuse mitigation scenarios.
  • Scalability of Adversarial Attacks: Adversarial attacks are costly and require domain expertise. While the study conducted four rounds, long-term performance could be examined with more extensive adversarial data collection.
  • Future Work: The authors suggest further research into the nuances and dynamics of abuse and counterspeech, exploring perspectivist frameworks for analyzing collective opinions on suitable replies, and investigating dynamic models that adapt to different domains for improved generation performance. They also highlight the need to address the challenges in generating counterspeech for specific targets like footballers.

Summary updated

BibTeX
@inproceedings{chung2024effectiveness,
  author = {Yi-Ling Chung and Jonathan Bright},
  title = {{On the Effectiveness of Adversarial Robustness for Abuse Mitigation with Counterspeech}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2024},
  doi = {10.18653/v1/2024.naacl-long.386},
}
Figure