TW-NLP at SemEval-2024 Task10: Emotion Recognition and Emotion Reversal Inference in Multi-Party Dialogues.

Wei Tian, Peiyu Ji, Lei Zhang, Yue JianNAACL 2024

TL;DR

This paper tackles the complex task of understanding emotions in multi-party, multilingual conversations, specifically focusing on recognizing emotions and identifying the "triggers" that cause emotional shifts. They introduce two novel models, MBERT-TextRCNN-PL for emotion recognition and XGBoost-EC for emotion reversal reasoning, achieving top rankings in the SemEval-2024 Task 10.

What Problem Does This Paper Address?

Understanding human emotions in conversations is crucial for building more empathetic and effective AI systems. However, multi-party dialogues, especially those involving multiple languages (like Hindi and English), present significant challenges. Emotions are not static; they evolve, and identifying why an emotion changes (the "emotion reversal inference") is as important as recognizing the emotion itself. This paper addresses the need for models that can accurately:

  1. Recognize emotions expressed by different speakers in mixed-language, multi-party conversations.
  2. Identify the specific utterances (triggers) that lead to a change in a speaker's emotional state.

Existing methods often struggle with the nuances of dialogue context, multilingualism, and the dynamic nature of emotional shifts.

Methods

The authors propose two distinct models for the two core tasks:

  • MBERT-TextRCNN-PL (for Emotion Recognition - Task 1): This model combines several powerful techniques:
  • MBERT: A multilingual BERT model is used as the backbone to capture semantic understanding across different languages.
  • TextRCNN: A Text Convolutional Neural Network is employed to capture local features and contextual information within utterances.
  • Prompt Learning: This technique is used to guide the MBERT model to focus on specific aspects of emotion recognition, effectively framing the input for better performance.
  • FGM (Fast Gradient Method): Integrated to enhance the model's generalization ability. The model concatenates features from MBERT and LSTM (which processes the sequence of MBERT outputs) to create a rich representation before final classification.

  • XGBoost-EC (Emotion Capturer) (for Emotion Reversal Inference - Tasks 2 & 3): This model is designed to identify emotion triggers:

  • Emotion Windows: Dialogues are segmented into fixed-size windows.
  • Emotion Encoding: Emotions within these windows are encoded numerically. Missing emotions are represented by -1.
  • XGBoost Classifier: A powerful gradient boosting algorithm is used to classify these windows and identify potential emotion-triggering utterances. This approach allows the model to learn patterns associated with emotional shifts based on historical context within the windows.

Datasets

The research utilizes datasets specifically curated for the EDiReF shared task at SemEval-2024:

  • MASAC-ERC Dataset (Task 1): Extracted from Indian television dramas, containing mixed Hindi-English dialogues with 8 emotion categories.
  • MASAC-EFR Dataset (Task 2): A larger dataset for emotion flipping reasoning in mixed Hindi-English dialogues.
  • MELD-EFR Dataset (Task 3): A dataset for emotion flipping reasoning in English dialogues.

The paper provides detailed statistics on the number of dialogues and utterances for training, validation, and testing for each dataset.

Key Insights and Ideas

  • Multilingualism is Key: The success of MBERT-TextRCNN-PL highlights the importance of using pre-trained multilingual models to handle code-switched conversations effectively.
  • Prompting for Focus: Prompt learning proves to be an effective strategy for directing large language models towards specific downstream tasks like emotion recognition, improving their performance without extensive task-specific fine-tuning.
  • Contextual Windows for Dynamics: The XGBoost-EC model's use of "emotion windows" is a clever way to capture the temporal dynamics of emotions. By looking at a sequence of utterances and their associated emotions, the model can better infer what caused a change.
  • XGBoost for Efficiency: Leveraging XGBoost for the emotion reversal task demonstrates its capability in handling structured data (encoded emotions and features) efficiently and effectively, outperforming other methods.

Key Results

The proposed models achieved impressive results in the SemEval-2024 Task 10:

  • Task 1 (Emotion Recognition): The MBERT-TextRCNN-PL model secured 6th place on the official leaderboard.
  • Task 2 (Emotion Flipping Reasoning): The XGBoost-EC model achieved 1st place, demonstrating its superior ability to identify emotion triggers.
  • Task 3 (Emotion Flipping Reasoning): The XGBoost-EC model also performed strongly, securing 5th place.

The paper notes that their MBERT-TextRCNN-PL model showed competitive performance against several strong baselines in Task 1, and the XGBoost-EC model significantly outperformed others in Task 2 and Task 3.

Limitations and Future Work

  • Limitations: While the models show strong performance, the paper doesn't extensively discuss specific limitations of their approaches, such as potential biases inherited from pre-trained models or the computational cost of processing long dialogues. The reliance on fixed-size windows in XGBoost-EC might miss triggers that span across window boundaries.
  • Future Work: The authors suggest their work provides a solid foundation for further research in multilingual emotion understanding and emotion reversal inference. Future directions could involve exploring more sophisticated methods for capturing long-range dependencies in dialogues, developing more robust multilingual emotion recognition models, and investigating the interpretability of emotion trigger identification.

Summary updated

BibTeX
@inproceedings{tian2024tw,
  author = {Wei Tian and Peiyu Ji and Lei Zhang and Yue Jian},
  title = {{TW-NLP at SemEval-2024 Task10: Emotion Recognition and Emotion Reversal Inference in Multi-Party Dialogues.}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2024},
}

Loading related content...

Figure