TL;DR
This paper tackles the challenge of detecting the emotion of "disgust" in Greek parliamentary speeches, a task that's surprisingly difficult but crucial for understanding political discourse. By developing a new Greek dataset and fine-tuning advanced language models, they achieve high accuracy for disgust detection and reveal how this emotion, and stigmatizing language, has evolved in Greek politics over three decades.
What Problem Does This Paper Address?
Sentiment analysis, which broadly categorizes text as positive or negative, is a well-established field. However, it often lacks the nuance needed to understand specific emotions, which can have distinct implications. For instance, "fear" and "anger" are both negative but imply different psychological states and potential actions. This paper focuses on "disgust," an emotion identified as particularly prevalent in electoral data, and its role in political discourse. The authors highlight a scarcity of resources for emotion detection in under-represented languages like Greek. They aim to bridge this gap by creating a new Greek dataset and demonstrating that specialized emotion detection, particularly for disgust, can be highly accurate and reveal subtle shifts in political language, including the use of stigmatizing terms.
Methods
The core of the methodology involves leveraging state-of-the-art Transformer-based language models (LLMs). They fine-tune both multilingual models (like XLM-R) and a Greek-specific model (GreekBERT) on their newly created datasets. A key aspect is the "transfer learning" approach: first fine-tuning on a large English emotion dataset (SE+) and then further fine-tuning on their Greek datasets (ART and PALO.GR). This allows them to benefit from general language understanding before specializing in Greek. For analyzing the evolution of language, they develop an "emotion context shift" algorithm. This algorithm scores sentences for disgust and then averages these scores over time (decades) and across specific target words, allowing them to track how the emotional context of words changes.
Datasets
- PALO.ES: A subset of Greek tweets annotated for ten emotion classes, used as an evaluation set.
- PALO.GR: Another subset of Greek tweets annotated for emotion, used for training.
- ART: A dataset of Greek tweets used for augmenting the training data.
- SE.EN: An English emotion detection dataset from the "1st SemEval E-c Task," used for initial fine-tuning of multilingual models.
- SE+: An enriched version of SE.EN, with added neutral tweets to better represent the "none" emotion class.
- Greek Parliament Corpus: A large corpus of Greek parliamentary speeches from 1989-2020, used for the primary analysis of emotion evolution and stigmatizing language.
Key Insights and Ideas
- Disgust is a surprisingly detectable emotion: While general emotion classification can be challenging, the paper demonstrates that "disgust" can be detected with high accuracy, comparable to sentiment and subjectivity classification. This suggests it's a distinct and identifiable emotional signal in text.
- Transfer learning is crucial for low-resource languages: Fine-tuning multilingual models first on a large English dataset before adapting them to Greek significantly improves performance, highlighting the effectiveness of this strategy for languages with fewer NLP resources.
- Political language evolves, and disgust plays a role: The study reveals distinct peaks in disgust emotion in Greek parliamentary records, often correlating with significant political events. This indicates that emotion detection can be a valuable tool for historical and political analysis.
- Stigmatizing language is on the rise: The "emotion context shift" analysis shows that terms like "handicapped" and "disability" are increasingly used in a negative, disgusted context in parliamentary speeches, suggesting a potential increase in stigmatizing language over time. The term "crazy" also appears to be used in a stigmatizing manner.
Key Results
- The best-performing models (X:ART+PALO, X:PALO, BERT:PALO) achieved high Area Under Precision-Recall Curves (AUPRC) for disgust detection, with X:PALO being slightly preferred for this study.
- The analysis of the Greek Parliament Corpus identified specific periods with high disgust scores, including 1991-1993, 2000, 2015, and 2019, often linked to significant political events (e.g., Macedonia naming dispute, elections).
- Political parties were ranked by their average disgust score, with far-right parties showing the highest scores, followed by the Democratic Social Movement and the Communist Party of Greece.
- The "emotion context shift" analysis revealed a statistically significant increase in the negative emotional context of "handicapped" and "disability" after 2011. The term "crazy" was also found to be used in a stigmatizing context.
Limitations and Future Work
- Text Length: The Transformer models used have limitations in handling very long texts, which could be beneficial for analyzing full parliamentary speeches. Future work could explore models like Longformer.
- Irony and Sarcasm: The current emotion classification schema does not explicitly account for irony or sarcasm, which are common in political discourse and can affect emotion detection.
- Out-of-Distribution Generalization: Emotion distribution can shift in political corpora over time, potentially affecting classifier performance. Investigating the out-of-distribution generalization ability of emotion classifiers is suggested.
- Broader Stigma Analysis: The authors plan to conduct a more thorough analysis of the term "crazy" and explore shifts in other emotions and potentially stigmatized words.
- Multilingual Application: Applying the method to more languages is a proposed future direction.
- Methodological Improvements: Experimenting with more augmentation strategies, additional annotators, and exploring topic extraction to correlate with emotions are also suggested.
Summary updated