Abstract
This paper introduces BHAAV (भाव), the first and largest Hindi text corpus designed for analyzing emotions expressed by writers through characters in stories. The corpus comprises 20,304 sentences extracted from 230 short stories across 18 diverse genres, such as inspirational and mystery. Each sentence was annotated into one of five emotion categories—anger, joy, suspense, sad, and neutral—by three native Hindi speakers. This resource aims to bridge the gap in Natural Language Processing (NLP) for Hindi, a widely spoken but historically low-resource language in the context of fine-grained emotion analysis.
Introduction
While sentiment analysis (classifying text as positive, negative, or neutral) is well-established in English NLP, emotion analysis remains underdeveloped for Hindi. Hindi is one of the top five most spoken languages globally, yet it is often categorized as "low-resource" due to a lack of large-scale, high-quality annotated datasets.
Existing Hindi datasets are typically small and focused on sentiment rather than the nuanced emotional states required for advanced applications like empathetic chatbots or automated storytelling. Hindi presents unique linguistic challenges, including free word order, rich morphology where verbs change based on subject gender, and significant spelling variations. BHAAV addresses these challenges by providing a robust benchmark for five core emotions.
The BHAAV Corpus
The BHAAV corpus (meaning "emotions" or "feelings" in Hindi) was constructed using a context-first approach. Researchers extracted over 20,000 sentences from 230 short stories. Unlike datasets that annotate sentences in isolation, BHAAV annotators were encouraged to consider the entire narrative arc to capture implicit emotions that depend on plot context.
The dataset distribution reflects real-world storytelling: approximately 11,700 neutral sentences, 3,100 sad, 2,400 joy, 1,500 suspense, and 1,400 anger. This variety allows for a more granular understanding of narrative flow compared to traditional sentiment datasets.
Methodology and Validation
Three native Hindi speakers with at least ten years of formal education annotated the corpus. To ensure the quality of these annotations, the researchers measured inter-annotator agreement using Cohen’s Kappa (κ) and Krippendorff’s Alpha (α) metrics.
The high agreement scores (averaging ~0.80) indicate a reliable dataset. The researchers established baselines using both traditional Machine Learning (SVM, Logistic Regression) and Deep Learning (CNN, Bi-LSTM) models, utilizing Fasttext embeddings trained on Hindi Wikipedia to account for morphological richness.
Key Insights and Results
Several critical insights emerged from the development of BHAAV:
- Contextual Dependency: Sentences that appear linguistically neutral can carry heavy emotional weight depending on the plot. BHAAV is specifically designed to capture this "narrator's perspective."
- The Difficulty of Suspense: Suspense is a unique addition to this corpus. It proved to be the most challenging emotion for both human annotators and machine models to categorize, often being confused with "Neutral" or "Joy."
- Model Performance: Logistic Regression emerged as the top performer with an accuracy of 0.62 and a Macro F1-score of 0.58. Interestingly, deep learning models performed slightly worse, likely because the dataset size is still relatively modest for training complex architectures from scratch without pre-existing large-scale Hindi language models.
Conclusion and Future Work
BHAAV represents a significant step forward for Hindi NLP, moving beyond simple sentiment to complex emotion analysis. While the corpus currently faces challenges related to class imbalance—a natural result of raw storytelling—it provides a vital foundation for future research. Future efforts will focus on expanding the corpus, developing models that better identify the "Primary Target of Opinion" across long-range text, and integrating these emotional insights into Text-to-Speech (TTS) systems to enable more expressive AI-driven narration.
Summary updated