Introduction
Detecting hate speech on Online Social Media (OSM) is a persistent "cat-and-mouse" game that becomes significantly more complex in multilingual societies like India. This paper details the approach taken by the PreCog IIIT Hyderabad team for the HASOC 2021 challenge, which focused on identifying hate speech in English, Hindi, Marathi, and code-mixed Hindi-English. The team's strategy, centered on an ensemble of multilingual transformers augmented with specialized classification heads, earned them a 3rd place overall ranking across six subtasks.
Problem Statement
The authors identify three primary hurdles in modern hate speech detection:
- Low-Resource Languages: While English is well-supported, regional languages like Marathi lack extensive datasets.
- Code-Mixing and Script Switching: The common practice of mixing languages (e.g., Romanized Hindi) often bypasses standard monolingual models.
- Conversational Context: Hate speech is frequently contextual; a reply may only appear toxic when viewed alongside its parent post.
Methodology
The proposed pipeline moves beyond standard transformer implementations by fusing deep learning with linguistic feature engineering.
The CNN Head Strategy
A core innovation was the replacement of the standard [CLS] token classification with a custom "CNN Head." The team extracted the last four hidden layers of XLM-RoBERTa (XLM-R), concatenated them, and passed the result through a Convolutional Neural Network (CNN). This architecture, shown in Figure 1, allows the model to capture local n-gram patterns that often signal offensive language more effectively than simple linear layers.
Handling Class Imbalance and Context
For fine-grained tasks where certain hate categories are rare, the team utilized Focal Loss instead of standard Cross-Entropy. This approach, illustrated in Figure 2 using a Multilingual MiniLM backbone, forces the model to prioritize "hard" or frequently misclassified examples during training.
To address conversational context, inputs were formatted as [CLS] <Tweet> [SEP] <Parent Context> [SEP], explicitly modeling the relationship between a post and its predecessor. Additionally, manual features such as profanity percentage, sentiment polarity (via TweetEval), and punctuation distribution were integrated into the final classification.
Datasets and Experimental Setup
The study utilized the HASOC 2021 dataset, covering binary and fine-grained classification for English, Hindi, and Marathi, as well as binary classification for Hindi-English code-mixed conversational threads. Interestingly, the authors found that incorporating historical HASOC data actually degraded performance, likely due to shifts in data distribution over time.
Results and Key Insights
The team's methods proved highly effective, particularly the CNN-augmented XLM-R model.
| Language | Task | Best Method | Accuracy | Macro F1 |
|---|---|---|---|---|
| English | Subtask 1A | XLM-R + CNN + Sentiment | 81.03% | 0.79 |
| Hindi | Subtask 1A | XLM-R + CNN | 80.09% | 0.77 |
| Marathi | Subtask 1A | XLM-R + CNN | 88.64% | 0.87 |
| Hi-En Code-mix | Subtask 2 | XLM-R with Norm | 69.36% | 0.70 |
Key takeaways from their experiments include:
- Hashtags as Noise: Removing hashtags improved performance, as they often acted as distractors.
- Emoji Utility: While emojis didn't impact binary detection, they were crucial for fine-grained classification, providing subtle sentiment cues.
- CNN Superiority: Using a CNN to process transformer embeddings consistently outperformed simple Multi-Layer Perceptron (MLP) heads by better identifying "trigger" phrases.
Conclusion
The PreCog team's success highlights the importance of architectural customization and specialized loss functions in multilingual hate speech detection. Future work will focus on domain-adaptive pre-training on social media corpora and exploring multi-task learning frameworks to create a more unified, efficient detection system.
Summary updated