NITK-IT_NLP@TamilNLP-ACL2022: Transformer based model for Toxic Span Identification in Tamil

Hariharan LekshmiAmmal, Manikandan Ravikiran, Anand Kumar MadasamyACL 2022

TL;DR

This paper tackles the challenge of identifying specific "offensive spans" within Tamil text, moving beyond simple classification of entire comments. They fine-tuned multilingual transformer models, with MuRIL achieving the best performance, demonstrating the effectiveness of specialized Indian language models for this nuanced task.

What Problem Does This Paper Address?

Content moderation on social media is a massive undertaking, and simply classifying an entire comment as "offensive" or "not offensive" isn't always enough. Identifying the exact words or phrases that constitute the offensive content is crucial for effective moderation, especially for smaller firms that can't afford extensive human moderation. This is particularly challenging in multilingual contexts, like India, where code-mixing (blending multiple languages within a single utterance) is common. This paper focuses on "Offensive Span Identification" in Tamil, aiming to pinpoint these harmful text segments.

Methods

The authors explored several multilingual transformer-based models for this task. They fine-tuned three pre-trained models: Multilingual BERT (M-BERT), MuRIL (specifically trained for Indian languages), and ELECTRA. After training, they employed Local Interpretable Model-Agnostic Explanations (LIME) to understand which parts of the text contributed most to the offensive classification. This allows for a more interpretable identification of the offensive spans.

Datasets

The primary dataset used for this shared task was the DravidianLangTech @ACL-2022 dataset, which included both supervised (annotated) and semi-supervised (unannotated) data for offensive span identification in Tamil. This dataset was derived from YouTube comments and contained code-mixed Tamil. Additionally, they utilized the HASOC-2021 shared task dataset, which provided 4000 comments with offensive and non-offensive labels, for training.

Key Insights and Ideas

  • Beyond Classification: The paper highlights the critical need to move beyond simple offensive/non-offensive classification to identifying the specific spans of text that are offensive. This provides more actionable information for content moderators.
  • Code-Mixing is Key: The prevalence of code-mixing in Indian social media necessitates models that can handle linguistic diversity.
  • Specialized Models Shine: MuRIL, a model specifically trained with multilingual representations for Indian languages, outperformed general multilingual models like M-BERT, underscoring the value of domain-specific pre-training.
  • Interpretability Matters: Using LIME to explain model predictions is crucial for understanding why a span is flagged as offensive, building trust and aiding in moderation strategy.

Key Results

The fine-tuned MuRIL model achieved the best overall character F1-score of 0.4489 on the offensive span identification task. This result placed their submission in first position for the shared task. The paper also includes examples of LIME interpretations, visually demonstrating how the model identifies offensive and non-offensive spans within comments.

Limitations and Future Work

The authors acknowledge that Offensive Span Identification is still in its developing stages. While their MuRIL model performed well, they express a desire to explore more on code-mixed data and develop improved solutions to this problem in the future. The paper does not explicitly detail limitations of the LIME method itself, but the general challenges of interpretability methods could be considered.

Summary updated

BibTeX
@inproceedings{lekshmiammal2022nitk,
  author = {Hariharan LekshmiAmmal and Manikandan Ravikiran and Anand Kumar Madasamy},
  title = {{NITK-IT\_NLP@TamilNLP-ACL2022: Transformer based model for Toxic Span Identification in Tamil}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2022},
}
Figure