Emotion Identification for French in Written Texts: Considering Modes of Emotion Expression as a Step Towards Text Complexity Analysis

Aline Étienne, Delphine Battistelli, Gwénolé LecorvéACL 2024

TL;DR

This paper introduces a novel approach to identifying emotions in French written texts, moving beyond basic emotion categories to consider how emotions are expressed (modes) and their complexity. They developed a new dataset and a fine-tuned CamemBERT model that outperforms existing methods, offering a more nuanced understanding of emotional expression in text.

What Problem Does This Paper Address?

While Natural Language Processing (NLP) has made strides in emotion detection, most research focuses on conversational or spoken data, and often simplifies emotions into basic categories. This paper tackles the under-explored area of emotion analysis in non-conversational written texts (like novels or news articles). The authors argue that understanding the modes of emotion expression (direct vs. indirect) and the type of emotion (basic vs. complex) is crucial for a deeper analysis of text complexity, particularly for understanding how children process narratives. Existing models often overlook these nuances, focusing solely on explicit emotional lexicons.

Methods

The core of this work is a multi-task learning approach using a fine-tuned CamemBERT model. This model is trained to perform four tasks simultaneously:

  1. Presence of Emotion (Task A): A binary classification to determine if a sentence contains emotional information.
  2. Mode of Expression (Task B): Identifying how the emotion is conveyed, categorized into four modes:
  3. Labeled: Directly stated emotion (e.g., "happy").
  4. Behavioral: Describing emotional behavior (e.g., "crying").
  5. Displayed: Heterogeneous linguistic features reflecting emotional state (e.g., interjections).
  6. Suggested: Emotion implied by a situation (e.g., reuniting with a friend suggests joy).
  7. Type of Emotion (Task C): Distinguishing between basic and complex emotions.
  8. Emotional Category (Task D): Classifying emotions into 11+1 categories (6 basic, 5 complex, and "other").

The CamemBERT model, a robust French language model, was fine-tuned on their custom dataset. They also compared their model against SVM, XGBoost, and GPT-3.5 (using in-context learning).

Datasets

The authors introduce a new dataset of French texts annotated for emotion presence, mode of expression, type, and category. This dataset comprises 1,594 French texts (28K sentences, 515K words) primarily sourced from journalistic texts (91%), with some encyclopedic articles (9%) and novels (1%). The data is intended for children aged 6-14 years and was annotated by six experts.

Key Insights and Ideas

  • Modes Matter: The distinction between direct (labeled) and indirect (behavioral, displayed, suggested) modes of emotion expression is a significant factor in text complexity and understanding. Indirect modes are considerably harder to detect.
  • Complexity is Multi-faceted: Emotion complexity isn't just about the category (basic vs. complex) but also how it's conveyed. Suggested emotions, for instance, are inherently more complex to interpret.
  • CamemBERT's Strength: Fine-tuning a specialized model like CamemBERT on a multi-task objective significantly outperforms general-purpose models like GPT-3.5 (in its few-shot prompting mode) for this nuanced task.
  • Human-Level Performance: Human evaluation suggests their model achieves performance comparable to human annotators, indicating a high level of generalization.

Key Results

  • The proposed CamemBERT model achieved significantly higher F1 scores across all tasks (B, C, and D) compared to SVM, XGBoost, and GPT-3.5. For tasks B, C, and D, their F1 scores were nearly double those of the best-ranked alternative models.
  • The model demonstrated strong performance, with F1 scores reaching up to 0.75 for Task A (presence of emotion), 0.64 for Task B (expression mode), 0.60 for Task C (emotion type), and 0.42 for Task D (emotional category).
  • Human evaluation showed high agreement rates between the model's predictions and expert annotations, suggesting the model's outputs are generally perceived as relevant.

Limitations and Future Work

  • Sentence-Level Focus: The current model operates at the sentence level, contextualized by adjacent sentences. This limitation means it might miss longer-range emotional connections or units that span across sentences.
  • Loss of Emotional Unit Granularity: Working at the sentence level can obscure the precise boundaries of emotional units within a sentence.
  • Future Directions: The authors plan to extend their work to an emotional unit level, remove contextual restrictions, and explore fine-tuning larger generative models. They also suggest investigating intra-sentential predictions and including notions like "experiencers." Ultimately, they aim to use this model for large-scale text complexity analysis and to further study the link between emotional language and psychological states.

Summary updated

BibTeX
@inproceedings{etienne2024emotion,
  author = {Aline Étienne and Delphine Battistelli and Gwénolé Lecorvé},
  title = {{Emotion Identification for French in Written Texts: Considering Modes of Emotion Expression as a Step Towards Text Complexity Analysis}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2024},
}
Figure