TL;DR
This document comprises the proceedings of the 5th Workshop on Online Abuse and Harms (WOAH 2021), a vital forum for researchers tackling the complex challenges of detecting, classifying, and mitigating online abuse. It showcases cutting-edge research from diverse fields, highlighting advancements in computational methods and interdisciplinary approaches to address the societal impact of digital harms.
What Problem Does This Paper Address?
The proliferation of digital technologies has brought immense benefits, but also amplified harmful and abusive behaviors online. Marginalized communities are disproportionately affected by issues like cyberbullying, hate speech, and interpersonal aggression. Existing automated systems for detecting such content often struggle with bias, brittleness, and a lack of transparency, especially when dealing with nuanced or context-dependent forms of abuse. This workshop aims to foster research that develops, interrogates, and applies computational methods to combat online abuse, emphasizing the need for interdisciplinary collaboration to address the underlying social questions of fairness and harm.
Methods
The papers presented at WOAH 2021 cover a wide range of methodologies, reflecting the interdisciplinary nature of the workshop. Key approaches include:
- Transformer-based Models: Several papers leverage advanced language models like BERT (e.g., HateBERT, VL-BERT) for tasks such as offensive language detection, profanity and hate speech modeling, and detecting hateful memes.
- Uncertainty Estimation: One paper explores using uncertainty estimation to improve model-moderator collaboration in content moderation.
- Fairness and Bias Mitigation: Techniques are presented for analyzing and improving the fairness of abusive language detection systems, including methods for correcting bias in lexicons and developing fair hate speech detection models.
- Multimodal Analysis: Research addresses the challenge of detecting hateful content in multimodal formats (e.g., memes) by developing models that jointly process text and images.
- Dataset Creation and Analysis: Several papers introduce new datasets or analyze existing ones to facilitate research in specific areas of online abuse, such as Dutch abusive language, Nepali social media, and political bias in German news.
- Invariant Rationalization: A method is proposed to mitigate biases in toxic language detection by learning invariant representations.
Datasets
The workshop features research utilizing and contributing to various datasets:
- Existing Datasets: Papers reference and build upon established datasets for tasks like offensive language detection, hate speech, and toxicity.
- New Datasets Introduced:
- Dutch Abusive Language Corpus (DALC): A resource for studying abusive language in Dutch.
- Nepali Social Media Datasets: Datasets for offensive language detection in Nepali.
- European Portuguese Lexicon for Minorities Related Terms (MIN_PT): A lexicon for minority-related terms in European Portuguese.
- Hateful Memes Datasets: Several papers focus on hateful memes, utilizing datasets from challenges like the Hateful Memes Challenge and introducing new ones for fine-grained detection and multimodal analysis.
- Targeted Insults and Compliments Dataset: A dataset designed to tackle online abuse by capturing both positive and negative targeted language.
- Contextual Dual-Annotated Dataset for In-Game Toxicity Understanding and Detection (CONDA): A dataset specifically for in-game toxicity.
Key Insights and Ideas
- The Nuance of Online Abuse: Detecting online abuse is not just about identifying keywords but understanding context, intent, and the interplay between different modalities (text, image). Hateful memes, for instance, often rely on the combination of seemingly innocuous text and images to convey harmful messages.
- Bias is Pervasive and Multifaceted: Bias in detection systems can manifest in various ways, including unfairness towards specific demographic groups, limitations in handling non-English languages, and inaccuracies in identifying protected groups. Addressing these biases requires careful dataset curation, model design, and evaluation.
- Interdisciplinary Collaboration is Crucial: Effectively combating online abuse requires insights from not only computer science (NLP, ML) but also social sciences, law, politics, and psychology. Understanding the social dynamics and ethical implications is as important as developing technical solutions.
- The Importance of Explainability and Transparency: The "black box" nature of many advanced models poses a challenge for trust and accountability. Research into explainability and improving model-moderator collaboration is vital for practical deployment.
- The Need for Robust Non-English Language Resources: A significant portion of online abuse occurs in languages other than English. Developing effective detection systems for these languages requires dedicated datasets and specialized models.
Key Results
- Improved Detection Performance: Several papers report state-of-the-art or significantly improved performance on various online abuse detection tasks using advanced models and novel techniques. For example, HateBERT demonstrates improved performance on abusive language detection in English.
- New Benchmarks and Datasets: The workshop saw the introduction of several new datasets and benchmarks, such as the DALC corpus and datasets for hateful meme detection, which will enable further research in under-resourced areas.
- Quantification of Bias: Studies provide quantitative evidence of bias in existing models and lexicons, highlighting specific areas for improvement.
- Effectiveness of Multimodal Approaches: Research on hateful memes demonstrates the effectiveness of multimodal approaches that jointly process text and images for better detection accuracy.
Limitations and Future Work
- Generalizability of Models: Models trained on specific datasets or languages may not generalize well to other contexts or languages, highlighting the need for more diverse and inclusive datasets.
- Data Scarcity for Low-Resource Languages: Many languages lack sufficient annotated data for training robust abuse detection systems. Future work should focus on developing methods for low-resource settings, such as transfer learning and cross-lingual approaches.
- Ethical Considerations: The ethical implications of deploying automated content moderation systems, including potential for censorship and algorithmic bias, remain a significant concern. Future research should continue to explore fairness, accountability, and transparency.
- Dynamic Nature of Online Abuse: Online abuse tactics evolve rapidly. Future work needs to develop adaptive systems that can keep pace with emerging forms of harmful content and communication strategies.
- Model-Moderator Collaboration: While progress has been made, optimizing the collaboration between AI systems and human moderators for effective and efficient content moderation is an ongoing area of research.
Summary updated