TL;DR
This paper tackles the challenge of identifying specific "toxic spans" within text, going beyond simply classifying a whole comment as toxic. The authors propose two methods: a supervised approach using SpanBERT for precise span detection and an unsupervised approach combining SVM with LIME for interpretability. Their SpanBERT model achieved a competitive F1 score of 67.84% in the SemEval-2021 competition.
What Problem Does This Paper Address?
The proliferation of user-generated content on social media necessitates effective moderation to foster healthy online discussions. While classifying entire comments as toxic is a step, it's often insufficient for human moderators who need to pinpoint the exact phrases or words causing toxicity. This is particularly challenging with lengthy comments where identifying specific toxic spans can be time-consuming. Existing toxicity detection methods often lack interpretability, providing a score without explaining why a text is deemed toxic. This paper addresses the need for precise, interpretable toxic span detection.
Methods
The authors present two distinct approaches:
- Supervised SpanBERT-based Model: This method leverages SpanBERT, a transformer-based model pre-trained to better understand and predict spans of text. Unlike BERT, SpanBERT masks contiguous spans and is trained with a span-boundary objective, allowing it to capture richer contextual information for span prediction. The task is framed as a sequence labeling problem, where tokens are classified as toxic or non-toxic. The model is fine-tuned on the toxic span detection dataset.
- Unsupervised SVM and LIME-based Model: This approach combines a linear Support Vector Machine (SVM) classifier with Local Interpretable Model-Agnostic Explanations (LIME). The SVM is trained on a large corpus of toxic and non-toxic comments to predict toxicity. LIME is then used to explain the SVM's predictions by highlighting words that contribute most to the toxicity classification. This allows for the identification of toxic spans by analyzing the explanations. Words with high influence scores are considered toxic, and their character offsets are retrieved.
Datasets
The primary dataset used is the SemEval-2021 Task 5: Toxic Spans Detection dataset, which includes a training set of 7939 toxic comments and a test set of 2000 toxic comments. For the unsupervised approach, they also incorporate a subset of the Kaggle Jigsaw Toxic Comment Classification Challenge dataset, adding 162,257 comments (16,225 toxic and 143,346 non-toxic) to their training data.
Key Insights and Ideas
- SpanBERT's Advantage: The paper highlights SpanBERT's effectiveness in toxic span detection due to its architecture, which is specifically designed to handle spans of text rather than just individual tokens. This pre-training objective allows for better representation and prediction of toxic segments.
- Interpretability Matters: The unsupervised LIME-based approach underscores the importance of interpretability. By using LIME, the authors can not only identify toxic spans but also understand why certain words are flagged, which is crucial for human moderators.
- Thresholding for Precision: In the LIME-based method, a thresholding technique is applied to filter words contributing to toxicity. This helps refine the identified spans and discard less influential words, leading to more precise toxic span detection.
- Supervised Outperforms Unsupervised: The results indicate that the supervised SpanBERT model generally outperforms the unsupervised SVM-LIME approach, suggesting that leveraging labeled data for span-specific training is more effective for this task.
Key Results
The supervised SpanBERT-based model achieved an F1 score of 67.84%, ranking 22nd out of 85 participants in the SemEval-2021 Task 5 competition. The unsupervised SVM-LIME method, with an optimal threshold of 0.12 or 0.13, achieved F1 scores around 0.59%. The paper also notes that SpanBERT outperformed other transformer-based models like BERT-base and BERT-large in their experiments.
Limitations and Future Work
The authors acknowledge that the unsupervised approach, while interpretable, did not achieve the same performance as the supervised method. They suggest that future work could focus on improving the performance of toxic spans detection tasks, potentially by exploring more advanced unsupervised techniques or hybrid approaches. The paper also mentions filtering predicted spans by removing toxic character offsets with a size of one, implying that single-character toxic spans might be less reliable or harder to detect accurately.
Summary updated