Detecting Offensive Tweets in Hindi-English Code-Switched Language

Puneet Mathur, Rajiv Ratn Shah, Ramit Sawhney, Debanjan Mahata

Abstract

Detecting offensive content in code-switched languages like "Hinglish" (Hindi-English) presents significant challenges due to a lack of formal structure, phonetic variation, and data scarcity. This paper introduces the Hindi-English Offensive Tweet (HEOT) dataset and proposes a Ternary Trans-CNN model. By leveraging transfer learning from a large English dataset to a smaller Hinglish target, the model achieves an accuracy of 83.9%, significantly outperforming models trained on Hinglish data alone.

Introduction

Social media moderation in linguistically diverse regions like India is complicated by the prevalence of "Hinglish"—a hybrid where Hindi words are written in Roman script and mixed with English. Standard NLP tools often fail because Hinglish lacks fixed grammar and exhibits high phonetic variation (e.g., "kutiya" vs "kutiyaa"). Furthermore, while English offensive speech datasets are abundant, high-quality annotated datasets for code-switched languages are rare. This study addresses these gaps by creating a specialized dataset and a robust classification model to distinguish between non-offensive, abusive, and hate-inducing content.

Methods

The researchers developed a Ternary Trans-CNN model utilizing a two-stage pipeline designed to bridge the gap between English and Hinglish semantics.

Preprocessing and Translation

To normalize the "wild" nature of Hinglish, the authors implemented a pipeline to clean URLs and mentions, and replace hashtags/emoticons with text. Crucially, a dictionary-based approach was used to translate Hinglish words into English equivalents, aligning Hinglish semantics with standard English vector spaces.

Transfer Learning Architecture

The model architecture consists of three 1D convolutional layers (with filter sizes 15, 12, and 10) followed by dense fully-connected layers, utilizing 200-dimensional GloVe embeddings. The transfer learning process involved training a source model on a large English dataset to learn general patterns of offensive language, then freezing the convolutional layers and fine-tuning the dense layers on the Hinglish HEOT dataset.

The Transfer Learning workflow, showing the transition from training on English Dataset A to fine-tuning on the Hinglish HEOT dataset.

Datasets

The study utilizes two primary datasets:

  • Dataset A: An English dataset from Davidson et al. (2017) containing 14,509 tweets.
  • HEOT Dataset: A novel dataset of 3,679 Hinglish tweets scraped via the Twitter API and manually annotated by ten researchers into three categories: non-offensive (1,414), abusive (1,942), and hate-inducing (323).

Results and Discussion

The core finding is the existence of "semantic parallelism"—insults in Hindi, once transliterated, map to similar vector spaces as English insults. This allows the model to overcome the "cold start" problem typically associated with small, specialized datasets.

The impact of transfer learning (TFL) was substantial. Training on HEOT alone yielded only 58.7% accuracy, likely due to overfitting on the small sample size. However, with pre-training on English data, accuracy jumped to 83.9%, with a corresponding F1-score of 0.714.

Performance metrics comparing the baseline English model, Hinglish without transfer learning, and Hinglish with transfer learning (TFL).

Conclusion

The Ternary Trans-CNN demonstrates that features learned from English are highly transferable to the code-switched domain. While the model performs well, future work should address dataset imbalance in the "hate-inducing" category and explore multimodal approaches to capture broader context in social media interactions.

Summary updated

BibTeX
@inproceedings{mathur2018detecting,
  author = {Puneet Mathur and Rajiv Ratn Shah and Ramit Sawhney and Debanjan Mahata},
  title = {{Detecting Offensive Tweets in Hindi-English Code-Switched Language}},
  booktitle = {SocialNLP @ ACL},
  year = {2018},
  doi = {10.18653/v1/w18-3504},
}
Figure