Transfer Learning and Prediction Consistency for Detecting Offensive Spans of Text

Amir Pouran Ben Veyseh, Ning Xu, Quan Hung Tran, Varun Manjunatha, Franck Dernoncourt, Thien Huu Nguyen

Abstract

While standard NLP models are effective at classifying entire sequences as toxic, they often fail to provide the granular explanations required for effective content moderation. This paper introduces TPOSD (Transfer learning and Prediction consistency for Offensive Span Detection), a framework designed to pinpoint the exact words and phrases that render a text offensive. By leveraging multi-task transfer learning from sentiment analysis and a novel consistency regularization method based on Optimal Transport, TPOSD achieves state-of-the-art results on offensive span detection, particularly in scenarios where training data is limited.

Introduction

Offensive Span Detection (OSD) is a critical task for modern content moderation, moving beyond binary "toxic or not" labels to identify specific problematic segments. Unlike sequence classification, OSD requires word-level labels, which are expensive and time-consuming to produce. Consequently, existing datasets are often small, leading to models that struggle with generalization or produce inconsistent predictions for similar linguistic structures. TPOSD addresses these challenges by incorporating auxiliary knowledge and enforcing decision-boundary robustness.

Methodology

The TPOSD framework is built upon three primary pillars designed to enhance feature representation and model stability:

  1. Core Architecture: The model utilizes a frozen BERT-base encoder to generate contextualized word representations. These are processed by a BiLSTM and a Feed-Forward Neural Network (FFNN) to predict token labels in the BIO (Beginning, Inside, Outside) format.
  2. Auxiliary Opinion Word Extraction (OWE): Recognizing that offensive words are frequently a subset of subjective "opinion words," the authors employ transfer learning. A separate model pre-trained on Sentiment Analysis (SA) provides "soft labels" for the OSD data. If masking a specific word causes the SA model to shift its prediction toward "Neutral," that word is flagged as a likely opinion word, guiding the OSD model's focus.
  3. Consistency Regularization via Optimal Transport: To mitigate the effects of data scarcity, the model is encouraged to produce consistent outputs for similar inputs. The authors use Optimal Transport (Wasserstein distance) to measure and minimize the discrepancy between predictions of the most similar samples within a mini-batch, resulting in a more robust decision boundary.

Experiments and Results

The TPOSD model was evaluated on the SemEval 2021 Task 5 (Toxic Spans Detection) benchmark. As shown in the performance comparison below, TPOSD significantly outperforms standard baselines and previous state-of-the-art models.

Performance comparison of TPOSD against baseline models on the SemEval 2021 Task 5 dataset.

TPOSD achieved an F1-score of 69.79, surpassing the previous state-of-the-art (IITK) at 68.20 and a standard BERT-CRF baseline at 63.19. Ablation studies confirmed that the Opinion Word Extraction task provided the most significant performance gain, followed by the consistency regularization.

Qualitative Analysis

Qualitative case studies highlight TPOSD's ability to distinguish between general subjectivity and actual toxicity. While standard models like BERT-CRF often get distracted by non-offensive opinion words (e.g., "joke" or "strange"), TPOSD successfully filters these out to focus on truly toxic spans.

Qualitative comparison of span detection between BERT-CRF and TPOSD against gold labels.

Discussion and Conclusion

The success of TPOSD demonstrates that toxicity detection can be significantly improved by treating it as a specialized subset of subjectivity analysis. By using transfer learning to identify opinion words, the model effectively restricts its search space to the most relevant candidates. While the current approach depends on high-quality sentiment analysis models, it provides a powerful template for leveraging auxiliary tasks to solve low-resource sequence labeling problems. Future work may explore additional auxiliary tasks, such as emotion detection or linguistic intensity, to further refine detection accuracy.

Summary updated

BibTeX
@inproceedings{veyseh2022transfer,
  author = {Amir Pouran Ben Veyseh and Ning Xu and Quan Hung Tran and Varun Manjunatha and Franck Dernoncourt and Thien Huu Nguyen},
  title = {{Transfer Learning and Prediction Consistency for Detecting Offensive Spans of Text}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2022},
  doi = {10.18653/v1/2022.findings-acl.128},
}
Figure