MarathiEmoExplain: A Dataset for Sentiment, Emotion, and Explanation in Low-Resource Marathi

Anuj Kumar, Mohammed Faisal Sayed, Satyadev Ahlawat, Yamuna PrasadEMNLP 2025

TL;DR

This paper introduces MarathiEmoExplain, the first large-scale, publicly available dataset for sentiment, emotion, and natural language explanations in Marathi, a low-resource language. By combining sentiment and emotion analysis with human-like justifications, this work aims to advance interpretable affective computing for underrepresented languages.

What Problem Does This Paper Address?

Natural Language Processing (NLP) research has historically focused on high-resource languages like English and Hindi, leaving many languages, including Marathi, significantly underrepresented. This imbalance hinders equitable digital participation and the development of inclusive AI systems. While sentiment analysis has progressed, Marathi datasets are limited, lacking fine-grained emotional categorization and, crucially, explanations for predictions. This lack of interpretability makes it difficult to trust and understand AI decisions, especially in sensitive applications like social media analysis or public opinion mining. This paper addresses this gap by creating a dataset that not only labels sentiment and emotion but also provides natural language justifications for these labels, paving the way for more transparent and trustworthy NLP models for Marathi.

Methods

The authors propose a two-stage architecture for jointly performing sentiment and emotion classification, followed by justification generation.

  1. Classification Stage: A generalized BERT-based transformer encoder is fine-tuned in a multitask setup to predict both sentiment (positive, neutral, negative) and emotion (joy, disgust, anger, surprise, sadness, fear, neutral) labels simultaneously. This shared encoder is followed by two parallel output layers, one for each task.
  2. Justification Generation Stage: A BART-style multilingual encoder-decoder model generates natural language justifications. This model is conditioned on the input Marathi sentence along with its predicted sentiment and emotion labels. The inputs are concatenated using a templated prompt format. This two-stage approach decouples classification from explanation, aiming for accurate predictions while maintaining interpretability.

The dataset was constructed by extending the existing L3CubeMahaSent corpus. Annotations were performed by native Marathi speakers, with justifications generated by GPT-4 under a human-in-the-loop framework to ensure accuracy and contextual alignment.

Datasets

  • MarathiEmoExplain Dataset: A newly introduced dataset comprising 10,762 Marathi sentences. Each sentence is annotated with:
  • Sentiment (positive, negative, neutral)
    • Emotion (joy, anger, surprise, disgust, sadness, fear, neutral)
    • Natural language justification (in English) explaining the sentiment and emotion labels.
  • L3CubeMahaSent: The base corpus used for creating the MarathiEmoExplain dataset.

Key Insights and Ideas

  • The Importance of Explanations in Low-Resource Settings: The paper strongly advocates for the necessity of natural language explanations, especially for low-resource languages. These justifications are crucial for building trust, enabling debugging, and ensuring fairness, going beyond simple accuracy metrics.
  • Emotion Boundaries are Fluid: The inter-annotator agreement analysis revealed lower agreement for semantically overlapping emotion categories like surprise and fear. This highlights the inherent ambiguity in emotion expression and the challenge of fine-grained emotion classification.
  • Multilingual Transformers Excel: Transformer-based models, particularly large multilingual ones like XLM-R, significantly outperform classical machine learning baselines for both sentiment and emotion classification in Marathi. This underscores their ability to capture nuanced semantics even with limited language-specific pretraining.
  • Justification Generation is Challenging: While transformer models show promise in generating justifications, the ROUGE scores indicate that producing coherent, label-grounded explanations that perfectly align with human annotations is difficult, especially when justifications are generated from predicted labels.

Key Results

  • Classification Performance: XLM-R achieved the strongest sentiment classification results (75.33% accuracy, 75.29% F1-score) and the highest macro-F1 for emotion classification (50.24%). BERT-Multi showed the best weighted F1 for emotion (69.13%). Classical models performed substantially worse.
  • Justification Quality: IndicBART consistently outperformed BART-base and BART-large in justification generation, achieving the highest ROUGE-1 (25.44) and ROUGE-L (20.47) scores. This suggests that pretraining on Indian language corpora is beneficial for this task.
  • Per-Emotion Performance: XLM-R demonstrated superior performance across most emotions, including challenging ones like Anger, Disgust, Sadness, and Surprise, highlighting its advantage in capturing subtle contextual cues.

Limitations and Future Work

  • Limitations:
    • Experimental Breadth: The work acknowledges limitations in experimental breadth and benchmarking, suggesting scope for future exploration.
  • Emotion Classification Challenges: The dataset exhibits class imbalance for certain emotions (e.g., fear, sadness), and overlapping affective categories pose a challenge.
  • GPT-4 Justifications: While human-refined, GPT-4 generated justifications can sometimes be generic or hallucinate content.
  • Future Work:
    • Dataset Expansion: Expanding and balancing the dataset is recommended.
  • Model Exploration: Benchmarking a wider range of multilingual and multimodal models.
  • Unified Task: Exploring the automation of classification and justification generation as a single, unified task.
  • Human-Centric Evaluation: Incorporating human evaluation beyond automatic metrics to assess the faithfulness and quality of explanations.

Summary updated

BibTeX
@inproceedings{kumar2025marathiemoexplain,
  author = {Anuj Kumar and Mohammed Faisal Sayed and Satyadev Ahlawat and Yamuna Prasad},
  title = {{MarathiEmoExplain: A Dataset for Sentiment, Emotion, and Explanation in Low-Resource Marathi}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
}
Figure