SkoltechNLP at SemEval-2021 Task 5: Leveraging Sentence-level Pre-training for Toxic Span Detection

David Dale, Igor Markov, Varvara Logacheva, Olga Kozlova, Nikita Semenov, Alexander PanchenkoACL 2021

TL;DR

This paper tackles the challenge of identifying toxic "spans" (specific words or phrases) within sentences, a more granular approach than just classifying entire comments. The authors demonstrate that fine-tuning pre-trained language models like RoBERTa is a strong baseline, and that leveraging sentence-level toxicity labels from a larger dataset significantly improves performance, especially when the target dataset is small.

What Problem Does This Paper Address?

Online platforms grapple with the pervasive issue of toxic and offensive content. While classifying entire comments as toxic or non-toxic has been a focus, the ability to pinpoint which specific parts of a sentence are toxic is crucial for effective content moderation and for developing tools that can "detoxify" text by suggesting replacements for offensive words. This task, SemEval-2021 Task 5, frames toxic span detection as a sequence labeling problem, a departure from previous sentence-level classification. The core challenge is the limited size of the dataset specifically annotated for toxic spans, making it difficult for models to generalize well.

Methods

The SkoltechNLP team explored several approaches, all building upon the powerful RoBERTa language model:

  • RoBERTa Tagger (Baseline): A standard RoBERTa model fine-tuned directly on the toxic spans dataset for sequence labeling. This served as their initial strong baseline.
  • Leveraging Sentence-Level Labels: Recognizing the scarcity of token-level data, they investigated ways to incorporate a larger dataset (Jigsaw) labeled for sentence-level toxicity.
  • Pseudo-labeling: They used their RoBERTa tagger to predict toxic spans on the Jigsaw dataset and then used these "pseudo-labels" to further train the model.
  • Joint Training (RoBERTa Classifier + Tagger): They fine-tuned a RoBERTa model first for sentence classification on Jigsaw, and then for span detection on the target dataset. This involved separate output layers for each task but shared underlying layers.
  • Joint Token- and Sentence-Level Classification (Tagging Classifier): This was their novel architecture. It jointly trains a model to predict both token-level toxicity and sentence-level toxicity, with the sentence-level score being an aggregation of token-level scores. This model was fine-tuned on both Jigsaw and the toxic spans dataset.
  • Span Aggregation: For converting token-level predictions back to character-level spans, they experimented with different aggregation strategies (e.g., max, mean, naive Bayes) and found that a simple "no aggregation" strategy (labeling a token as toxic if its score exceeds a threshold) performed best.

Datasets

  • SemEval-2021 Task 5 Dataset: The primary dataset for training and evaluation, consisting of 7,940 English comments with manually annotated toxic spans at the character level. A development set of 690 texts and a hidden test set of 2,000 texts were also provided.
  • Jigsaw Toxic Comments Dataset: A much larger dataset (140,000 user utterances) labeled for sentence-level toxicity (toxic/safe). This was used for pre-training and transfer learning.

Key Insights and Ideas

  • Sentence-level data is a powerful signal for token-level tasks: Even though the Jigsaw dataset doesn't provide span-level annotations, using its sentence-level labels significantly boosted performance on the toxic span detection task, especially when the target dataset was small. This highlights the effectiveness of transfer learning from related, larger datasets.
  • Joint modeling can be beneficial: Their proposed "tagging classifier" architecture, which jointly models token- and sentence-level toxicity, achieved the best results, suggesting that these two levels of toxicity are complementary.
  • The "no aggregation" strategy for spans is surprisingly effective: Instead of trying to aggregate token scores to label entire words, simply thresholding individual token scores and then mapping them back to characters yielded the best results. This might prevent the propagation of errors when labeling words.
  • Vocabulary saturation is not the issue: Contrary to expectations, the size of the toxic vocabulary in the SemEval dataset did not appear to saturate with increasing data size. This implies that models will frequently encounter unseen toxic words, underscoring the need for robust generalization through pre-training.

Key Results

  • The SkoltechNLP team's best model, the "tagging classifier," achieved an F1-score of 0.683 on the test set, placing them among the top 20% of participants.
  • This was only 2 points below the best team's score (0.708), demonstrating the competitiveness of their approach.
  • The pseudo-labeling approach also performed well, achieving an F1-score of 0.682.
  • Simply fine-tuning RoBERTa on the toxic spans dataset (RoBERTa tagger) yielded an F1-score of 0.678.
  • Their baselines (e.g., Word-based LogReg, CRF, LSTM) performed significantly worse, with the best baseline achieving only 0.556, indicating the superiority of transformer-based models.
  • The paper showed that the benefit of sentence-level pre-training was most pronounced when the token-level training data was small (under 1,000 samples), becoming less significant with larger datasets.

Limitations and Future Work

  • Ambiguity in Annotations: The authors suggest that the inherent ambiguity in human annotations might limit the achievable performance on this task, even for the best models.
  • Overfitting to Test Set: They note that the small development set made it difficult to tune hyperparameters, leading them to explore different thresholds during the competition. They advocate for making test sets unavailable until the end of the competition to prevent such indirect overfitting.
  • Low-Resource Scenarios: While their models show improvement, they acknowledge that they might still be best suited for low-resource scenarios where extensive token-level annotations are unavailable.
  • Future Work: The paper doesn't explicitly detail future work, but the success of joint modeling and transfer learning suggests further exploration in these areas. Analyzing the impact of different aggregation strategies and exploring more sophisticated ways to combine sentence and token-level information could be fruitful.

Summary updated

BibTeX
@inproceedings{dale2021skoltechnlp,
  author = {David Dale and Igor Markov and Varvara Logacheva and Olga Kozlova and Nikita Semenov and Alexander Panchenko},
  title = {{SkoltechNLP at SemEval-2021 Task 5: Leveraging Sentence-level Pre-training for Toxic Span Detection}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2021},
}
Figure