TL;DR
This paper introduces UNITYAI-GUARD, a novel framework for detecting toxic content in six low-resource Indian languages. It addresses the critical gap in current NLP tools, which largely ignore these languages, by providing state-of-the-art models, a large manually verified dataset, and an accessible API.
What Problem Does This Paper Address?
The internet has become a global communication platform, but it's also a breeding ground for online toxicity, including hate speech, harassment, and abuse. While significant progress has been made in detecting this toxicity in high-resource languages like English, many other languages, particularly those spoken in India, remain underserved. India is incredibly linguistically diverse, with 22 official languages and hundreds of dialects. However, most of these languages are computationally "low-resource," meaning they lack sufficient digital data, annotated datasets, and pre-trained language models for tasks like content moderation. This creates a significant challenge for maintaining user safety and enabling effective content moderation across these diverse linguistic communities. Existing tools often fail to support these languages or rely on synthetically generated data, which doesn't capture the nuances of real-world toxic language.
Methods
UNITYAI-GUARD tackles this problem with a multi-pronged approach:
- State-of-the-Art Models: The framework utilizes and fine-tunes three differently sized language models (mBERT, llama-3.2-1B, and aya-expanse-8B) to achieve high performance in toxicity classification across the targeted Indian languages.
- Comprehensive Framework: It's not just about detection; UNITYAI-GUARD offers a user-friendly web portal with several key functionalities:
- Toxicity Classification: The core function of identifying toxic content.
- Transliteration: Enables users to input text in their native script and see it transliterated, bridging script differences.
- Speech-to-Text: Enhances accessibility by allowing users to speak their input.
- API Access: Provides programmatic access for developers to integrate the toxicity detection capabilities into their own applications.
- Manual Verification: A crucial aspect is the rigorous quality control of the dataset, involving manual verification by two native speakers for each language, ensuring the authenticity and accuracy of the labeled data.
Datasets
The paper introduces a newly curated dataset for six low-resource Indian languages: Hindi (hi), Telugu (te), Marathi (mr), Urdu (ur), Tamil (ta), and Punjabi (pa).
- Training Data: 567,651 instances.
- Test Data: 30,000 instances, which were manually verified.
- Distinction: This dataset stands out due to its emphasis on manual verification, aiming to overcome the limitations of synthetically generated data found in some previous work. The dataset is planned to be released under the CC-BY-4.0 license.
Key Insights and Ideas
- The Importance of Manual Verification: The paper strongly emphasizes that manually verified data is crucial for accurately capturing the nuances of toxicity in low-resource languages, as synthetic data often falls short.
- Larger Models, Better Performance: The results indicate that larger language models (like aya-expanse-8b) demonstrate significantly better capability in processing and understanding diverse Indian scripts compared to smaller models or zero-shot approaches.
- Cross-Lingual Nuances are Key: The authors highlight that simply applying a unified model across all languages is insufficient. They employed language-specific parameterization to account for variations in how certain terms or phrases can be perceived differently across linguistic and cultural contexts.
- Accessibility is Paramount: The inclusion of transliteration and speech-to-text features in the UI demonstrates a commitment to making toxicity detection tools usable and accessible to a wider range of users, not just those proficient in English or specific scripts.
Key Results
- Impressive F1-Score: UNITYAI-GUARD achieves an average F1-score of 84.23% across the six languages.
- Model Performance: The aya-expanse-8b model showed superior accuracy (87.21%) compared to mBERT (83.40%) in fine-tuned settings. F1 scores also consistently improved with model size, reaching up to 86.96%.
- Zero-Shot Performance: The zero-shot evaluation results were significantly lower, highlighting the necessity of fine-tuning for these low-resource languages.
- Dataset Size: The creation of the largest toxicity dataset for these languages, with 567,651 training and 30,000 manually verified test instances, is a significant contribution.
- User Engagement: Since its launch, the UI has recorded over 200 requests, with Hindi being the most frequently inferred language. Approximately 33% of responses were flagged as toxic.
Limitations and Future Work
- Limited Language Coverage: The current study is constrained to six languages due to the time and cost associated with manual annotation.
- Future Work: The authors plan to:
- Expand language coverage to include more Indian languages, particularly from Dravidian and North-Eastern families.
- Incorporate contextual understanding for culturally-specific nuances.
- Develop robust cross-lingual transfer approaches to handle India's script diversity.
- Release the complete dataset with fine-grained annotations across 14 distinct toxicity categories.
Summary updated