Introduction
Marathi, despite having over 83 million speakers, remains a "low-resource" language in the field of Natural Language Processing (NLP). The scarcity of high-quality annotated datasets presents two primary challenges: overfitting and domain sensitivity. Models trained on limited data often memorize specific phrases instead of learning generalizable sentiment patterns, leading to poor performance when applied to data from different domains—for instance, transitioning from Twitter-based datasets to Reddit comments. This study investigates the efficacy of various data augmentation strategies, including BERT-based masking and GPT-driven generation, to build more robust sentiment analysis models for Marathi.
Methodology
The researchers established a baseline using the marathi-bert-v2 model and evaluated several advanced data augmentation techniques designed to improve generalization.
BERT-based Masking Strategies
- Random Masking: 40% of words in a sentence were masked, with the model predicting replacements. This was tested in Sequential mode (predicting one word at a time) and Parallel mode (predicting all masked words simultaneously).
- Named Entity Masking: A targeted approach using Named Entity Recognition (NER) to identify and mask specific entities such as people, places, or organizations. This forces the model to focus on the linguistic context of sentiment rather than specific entity associations.
Generative and Semantic Approaches
- GPT-based Generation: Utilizing the
l3cube-pune/marathi-gptmodel for label generation (pseudo-labeling unlabeled data) and text completion (generating sentiment-consistent sentence endings). - Back-Translation & Paraphrasing: Standard techniques involving translation to English and back to Marathi, or using specialized Indic-language paraphrasing models to create semantic variations.
Datasets and Experimental Setup
The study utilized two primary datasets to evaluate cross-domain generalization:
- L3CubeMahaSent: Approximately 16,000 Marathi tweets covering politics and social issues.
- GoEmotions (Marathi): A Marathi translation of Reddit comments, simplified to a three-way sentiment scale (positive, negative, neutral).
By training on one domain and testing on the other, the researchers measured the model's ability to generalize beyond its training distribution.
Results and Discussion
The experimental results highlight the significant impact of data augmentation on model robustness, particularly in cross-domain scenarios.
Performance Gains
The most notable improvements were observed in cross-domain tests. For a model fine-tuned on GoEmotions (Reddit) and tested on MahaSent (Twitter), the baseline accuracy was 0.6882. Implementing Sequential Named Entity Masking increased this accuracy to 0.7362, demonstrating that masking entities helps the model learn structural sentiment cues rather than memorizing entity-sentiment correlations.
In-domain performance also saw gains. For models trained and tested within the MahaSent domain, Sequential Random Masking achieved a peak accuracy of 0.8430. Furthermore, GPT-based label generation proved highly effective, yielding an in-domain accuracy of 0.8435.
Sequential vs. Parallel Masking
Across nearly all tests, sequential masking outperformed parallel masking. This is likely because the model can use previously predicted words as context for subsequent predictions, resulting in more coherent and linguistically sound synthetic data.
Conclusion
This research demonstrates that advanced data augmentation techniques can significantly mitigate the challenges of low-resource NLP for Marathi. By focusing on context over specific content through methods like Named Entity Masking, models become more resilient to domain shifts. While currently limited by the scarcity of advanced preprocessing tools for Marathi, these strategies provide a scalable framework for improving sentiment analysis in other Indic and low-resource languages.
Summary updated