TL;DR
This paper introduces SemEval-2021 Task 5, focusing on identifying specific "toxic spans" within online comments that contribute to their overall toxicity. The task aims to move beyond simple toxic/non-toxic classification by pinpointing the exact problematic phrases, and the winning systems leveraged transformer-based models with ensemble techniques to achieve state-of-the-art results.
What Problem Does This Paper Address?
Online discussions are often plagued by toxic content, ranging from rudeness to hate speech. While classifying entire posts as toxic is a common approach, it's insufficient for effective moderation. Moderators need to quickly identify why a post is toxic, especially in long comments. This paper addresses the challenge of automatically highlighting the specific "toxic spans" – the exact words or phrases responsible for a post's toxicity – to assist human moderators and improve the granularity of toxicity detection systems.
Methods
The core of the task involved supervised sequence labeling, where models were trained to identify toxic spans. Participants explored various approaches, with a strong emphasis on transformer-based models like BERT and RoBERTa. Key techniques included:
- Fine-tuning Pre-trained Transformers: Models like BERT and RoBERTa were fine-tuned on the provided toxic span dataset.
- Sequence Labeling Architectures: Conditional Random Fields (CRFs) were frequently used on top of transformer outputs to predict token-level labels (e.g., beginning, inside, or outside a toxic span).
- Ensemble Methods: Combining multiple models or approaches (e.g., token labeling and span extraction, or different transformer layers) proved highly effective, with the top-ranked systems employing ensembles.
- Rationale Extraction: Some teams explored methods that identify toxic spans as explanations for a post's toxicity, even without explicit span annotations in their training data. This approach, while less successful in this competition, is highlighted as a promising direction for future work.
Datasets
The primary dataset for this task was created by re-annotating a subset of the Civil Comments dataset.
- Civil Comments Dataset: This dataset, originally collected by Civil Comments, contains over two million comments.
- Toxic Spans Dataset: A revised version of the annotation system was used to re-annotate approximately 30,000 comments previously identified as toxic by crowd raters. These annotations were at the character level and were used to create gold standard toxic spans. The dataset was split into trial, training, and test sets.
Key Insights and Ideas
- Granularity Matters: Moving from post-level toxicity classification to toxic span detection significantly increases the complexity but offers much greater utility for moderation.
- Ensembles Dominate: Combining multiple models, especially those with complementary strengths (like token labeling and span extraction), was a winning strategy.
- Transformer Power: Pre-trained transformer models, particularly RoBERTa, formed the backbone of most high-performing systems.
- The Challenge of No Toxic Spans: Systems struggled with posts that were labeled as toxic but contained no clearly identifiable toxic spans, suggesting this is a difficult edge case.
- Subjectivity and Annotation Challenges: The task's inherent subjectivity led to moderate inter-annotator agreement, and participants noted inconsistencies and errors in the annotations themselves.
Key Results
The competition saw 36 teams submit system descriptions.
- Winning System (HITSZ-HLT): Achieved the highest character-based F1 score of 70.83%. Their approach combined BERT with token labeling and span extraction, using an ensemble of these methods.
- Second Place (S-NLP): Scored 70.77% F1, also employing an ensemble of RoBERTa-based models.
- Benchmarks: The provided benchmarks (BENCHMARK I, II, III) scored 69.89%, 50.98%, and 12.22% respectively, highlighting the performance gap between sophisticated models and simpler baselines.
- Lexicon-based Systems: While scoring lower overall, lexicon-based approaches were noted for their efficiency and interpretability, with some achieving surprisingly competitive F1 scores.
Limitations and Future Work
- Annotation Subjectivity: The inherent subjectivity of "toxicity" and the identification of specific spans leads to challenges in annotation consistency and agreement.
- Posts Without Toxic Spans: Detecting toxicity in posts that are toxic overall but lack clearly defined toxic spans remains a significant challenge.
- Context Dependency: Long toxic spans are more context-dependent and harder to detect than short, common abusive words.
- Rationale Extraction Potential: While not the top-performing approach in this task, rationale extraction methods that don't require span-level annotations are a promising area for future research, especially for unsupervised or semi-supervised settings.
- Dataset Improvement: Future datasets could benefit from more annotators per post to improve reliability and address ambiguity, bias, and inconsistencies.
- Task Variations: Future competitions could explore variations like requiring post classification before span detection, or distinguishing between different types of toxicity (insult, threat, etc.) with their corresponding spans.
Summary updated