TL;DR
This paper introduces a novel human-in-the-loop (HITL) methodology for efficiently collecting high-quality, multi-target counter-narrative (CN) datasets to combat online hate speech. By iteratively refining a generative language model with expert-reviewed data, they significantly reduce the need for manual annotation and produce a unique, expert-based dataset.
What Problem Does This Paper Address?
Online hate speech is a pervasive issue that harms individuals and society. While moderating content is one approach, counter-narratives (CNs) – informed, non-aggressive responses using credible evidence and factual arguments – are emerging as a more constructive solution. However, creating large, high-quality datasets of hate speech (HS) and corresponding CNs for training AI models is challenging. Existing methods often struggle to balance data quantity and quality, leading to datasets that are either too small, lack diversity, or are not expert-verified. This paper aims to bridge this gap by developing a scalable and cost-effective HITL approach for generating such datasets.
Methods
The core of the methodology is an "author-reviewer" framework. An "author" module, powered by a GPT-2 language model fine-tuned on an initial seed dataset, generates candidate HS/CN pairs. These pairs are then reviewed and potentially post-edited by human annotators ("reviewers"). The key innovation lies in the iterative nature of this process: the post-edited data from each loop is fed back to retrain and fine-tune the author module, allowing it to learn from its mistakes and improve its generation quality over time.
The authors experimented with two main sessions:
- Session One: Focused on a standard HITL iterative process, where the model was retrained sequentially with accumulated data.
- Session Two: Explored various "author configurations" to address issues like target imbalance and improve novelty. These configurations involved conditioning the model on different inputs, such as specific hate target labels (V6,LAB), offensive speeches from another corpus (V6,SBF), or argumentative structures from debate platforms (V6,ARG), and a mixed approach (V6,MIX).
Datasets
- Seed Dataset (V₁): A dataset of 880 HS/CN pairs collected through "nichesourcing" from 20 experts at two NGOs. This dataset was curated for specific hate targets like race, religion, sexual orientation, disability, and gender.
- Newly Introduced Benchmark: The paper presents the CONAN dataset, a novel, expert-based, multi-target HS/CN dataset containing 5000 pairs. This dataset is unique in its expert validation and multi-target coverage.
Key Insights and Ideas
- Iterative Refinement is Key: The HITL approach, where the generative model is continuously retrained with expert-validated data, significantly improves data collection efficiency and the quality of generated pairs over time.
- Balancing Quality and Quantity: While iterative loops naturally increase data quantity, they can sometimes lead to a decrease in novelty and diversity. The second session's experiments with different author configurations were crucial for addressing this trade-off.
- The Power of Contextual Conditioning: Conditioning the language model on specific hate targets (V6,LAB) or relevant offensive language (V6,SBF) proved effective in improving data balancing and acceptance rates.
- Argumentation for Counter-Narratives: Using argumentative data (V6,ARG) showed promise in generating more diverse and potentially novel counter-narratives, though it required careful integration with HS/CN data.
- Human Oversight is Crucial: The authors emphasize that while AI can assist, human moderators are essential for final decision-making, especially in sensitive areas like countering hate speech.
Key Results
- The HITL methodology successfully generated 5000 HS/CN pairs.
- Acceptance Rate: Increased significantly across loops, with the V6 configurations achieving higher acceptance rates for both untouched and modified pairs compared to V5. V6,MIX achieved the highest acceptance rate for modified pairs (66.15%).
- HTER (Human-Targeted Editing Effort): Decreased across loops in Session One, indicating reduced reviewer effort. While HTER increased slightly in Session Two across V6 configurations, it remained below the acceptability threshold of 0.4. V6,ARG showed the lowest HTER.
- Novelty and Repetition Rate: While novelty and repetition rate generally decreased in Session One, Session Two configurations (especially V6,MIX) showed improvements in novelty.
- Target Imbalance: The initial dataset showed significant imbalance, with "MUSLIMS" being the most frequent target. Session Two configurations (V6,SBF, V6,LAB, V6,MIX) significantly improved target balancing, reducing the Imbalance Degree (ID) from 4.5 (V5) to 2.3 (V6).
- Qualitative Findings: The model demonstrated the ability to generate HS/CN pairs for unseen targets and even for fictional contexts (e.g., "muggleborns").
Limitations and Future Work
- Decreasing Novelty/Diversity: A consistent challenge across iterative loops was the gradual decrease in novelty and diversity of generated content. Session Two aimed to mitigate this, showing promising results with specific configurations.
- Potential for "Perfect" vs. "Sub-optimal" Content: The authors note that V6 configurations were better at producing "sub-optimal" but usable material, rather than consistently "perfect" responses.
- Future Work: The authors suggest exploring the use of argumentation models without fine-tuning on HS/CN data, investigating whether GPT-2 produces novel arguments or sophisticated paraphrases, and further research into generating HS/CN pairs for novel hate targets. They also highlight the potential of using these models as "suggesting tools" for NGOs.
Summary updated