TL;DR
This paper presents a system for detecting toxic spans of text in online content, combining multiple models to capture both individual toxic words and longer offensive phrases. While the ensemble approach showed promise on development data, its performance didn't fully transfer to the test set, highlighting the challenges of generalization in this domain.
What Problem Does This Paper Address?
The paper tackles the problem of toxic span detection, a crucial task in content moderation and online safety. Unlike simpler toxicity classification (which labels an entire text as toxic or not), this task requires pinpointing the exact phrases or words that are toxic. This is challenging because toxicity can manifest in various ways: single offensive words, subtle implications, or even entire sentences designed to be harmful. Existing methods often struggle to capture this nuance, leading to either over- or under-detection. The SemEval-2021 Task 5 specifically aimed to address this by requiring systems to identify these toxic spans with high precision.
Methods
The UAntwerp team developed a stacked generalization ensemble system. This means they trained several individual "component" models and then combined their predictions using another model. Their strategy involved two main types of component models:
- Binary Word Level Models: These models focused on predicting whether an individual word was toxic. They used sequences of words before and after the target word as features. Two specific implementations were used: a Siamese-LSTM with GloVe embeddings and an LSTM fine-tuned on BERT-base outputs.
- Categorical Span Based Models: These models aimed to predict toxic token labels for complete sequences. Three transformer-based models were fine-tuned for this: ALBERT, ROBERTa, and ELECTRA.
These five component models' predictions were then concatenated and fed into a Bidirectional LSTM (BiLSTM) for the final ensemble prediction. This approach was motivated by the hypothesis that models with different interpretations (word-level vs. span-level, different architectures) would have low correlation, making them ideal for ensemble components.
Datasets
The primary dataset used was the SemEval-2021 Task 5: Toxic Spans Detection dataset. This dataset consisted of:
- Development Data: 7939 text samples of varying lengths (up to 1000 characters).
- Test Data: 2000 text samples.
The task's evaluation metric was a pairwise F1-score of character offset predictions.
Key Insights and Ideas
- Toxicity is Multifaceted: The development data revealed that toxic spans are not just single words but can be longer phrases or even entire sentences, necessitating approaches that can capture both granular and contextual toxicity.
- Ensemble Power (with caveats): Combining diverse models (word-level and span-level, different architectures) is a common strategy for improving performance. The paper explored this by stacking transformer models and word-level models.
- Correlation Matters: The authors observed that model correlation played a role in ensemble accuracy. Models with low correlation (like BERT and GLOV) were seen as better ensemble components.
- Feature Impact: Error analysis indicated that longer toxic spans and spans with a higher frequency of stop words negatively impacted model performance.
Key Results
The UAntwerp system achieved an F1-score of 0.6755 on the test data, ranking 26th overall in the SemEval-2021 Task 5.
- On the development data, the ensemble model showed a 5% increase in F1-score compared to component models.
- However, this performance boost was not fully transferred to the test data, where the correlation between models increased on common examples, potentially limiting the ensemble's generalization.
- The transformer models (ALBERT, ROBERTa, ELECTRA) generally performed better than the word-based models.
- A simple lexical lookup baseline achieved an F1-score of 0.4161, and an SVM-based approach achieved 0.5489 on the test data.
Limitations and Future Work
- Generalization Gap: The most significant limitation noted is the performance drop from development to test data, suggesting the ensemble struggled to generalize to the test set's distribution, particularly regarding model correlation.
- Annotation Rationales: The paper acknowledges that multiple annotation rationales (single words vs. longer sentences) exist, and models were less accurate at detecting longer spans.
- Future Work: The authors suggest exploring strategies that incorporate model transferability in component model selection to better handle variations in annotation rationale and improve generalization. They also point to recent work in this area (Fortuna et al., 2021) as a potential direction.
Summary updated