E-ICL: Enhancing Fine-Grained Emotion Recognition through the Lens of Prototype Theory

Zhou Yang, Zhaochun Ren, Chenglong Ye, Yu‐Feng Wang, Haizhou Sun, Chao Chen, Xiaofei Zhu, Yunbing Wu, Xiangwen Liao

TL;DR

This paper introduces E-ICL, a novel method to improve fine-grained emotion recognition in Large Language Models (LLMs) by leveraging prototype theory. E-ICL addresses the limitations of standard In-Context Learning (ICL) by using emotionally similar examples with dynamic labels and an exclusionary prediction strategy, leading to more accurate and robust emotion detection.

What Problem Does This Paper Address?

Large Language Models (LLMs) have shown impressive capabilities in various tasks through In-Context Learning (ICL), where they learn from a few examples provided in the prompt. However, this approach struggles significantly with fine-grained emotion recognition. The paper identifies two key issues: 1) ICL relies on examples that are semantically similar but not necessarily emotionally accurate, leading to misinterpretations. 2) ICL is easily distracted by irrelevant emotion categories, hindering its accuracy and robustness, especially when dealing with nuanced emotions that are hard to distinguish.

Methods

The proposed method, Emotional In-Context Learning (E-ICL), tackles these problems by reframing ICL through the lens of prototype theory. E-ICL consists of four main steps:

  1. Emotion-Similar Example Retrieval: Instead of just semantic similarity, E-ICL retrieves examples that are emotionally similar to the query. This is achieved using an auxiliary emotion model (RoBERTa-large) to generate emotion vectors.
  2. Dynamic Soft Label Construction: Human emotions are complex and often blended. E-ICL moves beyond single, deterministic labels by creating "dynamic soft labels." These labels combine the ground-truth emotion with probabilities predicted by the auxiliary model for the top-k emotions, offering a more nuanced representation.
  3. Candidate Emotion Division: Inspired by human decision-making, E-ICL divides potential emotion categories into "possible" and "impossible" sets. This is done by leveraging the auxiliary model's predicted probabilities for the query.
  4. Exclusionary Emotion Prediction: The LLM is then prompted to predict the emotion, but it first considers the "possible" emotions and then the "impossible" ones. This exclusionary strategy helps the LLM focus on more likely candidates and avoid being misled by irrelevant categories.

Crucially, E-ICL is a plug-and-play method that uses an auxiliary model without requiring any additional training of the LLM itself.

Datasets

The paper evaluates E-ICL on four fine-grained emotion recognition datasets:

  • EDOS
  • Empathetic-Dialogues (ED)
  • EmpatheticIntent (EI)
  • GoEmotions (GE)

Key Insights and Ideas

  • ICL and Prototype Theory: The paper provides evidence that ICL, especially in emotion recognition, aligns with prototype theory. This means that the more similar an example is to a prototype (demonstration), the more accurately it can be recognized.
  • Semantic vs. Emotional Similarity: Simply retrieving semantically similar examples can lead to "emotion misunderstanding." E-ICL's focus on emotional similarity is key to overcoming this.
  • Dynamic Labels for Nuance: Human emotions are rarely black and white. The use of "dynamic soft labels" acknowledges this complexity and provides a richer representation than traditional one-hot labels.
  • Exclusionary Prediction for Robustness: Mimicking human decision-making by first eliminating unlikely options (exclusionary prediction) significantly improves accuracy and robustness, especially in tasks with many similar categories.
  • Auxiliary Model's Role: The auxiliary emotion model is vital for retrieving emotionally similar examples, generating dynamic labels, and dividing candidate emotions. Its performance can influence E-ICL, but the method shows robustness even when the auxiliary model is weaker.

Key Results

  • E-ICL consistently outperforms standard Zero-Shot and ICL baselines across all tested datasets, particularly on finer-grained emotion recognition tasks.
  • The method demonstrates significant performance improvements, with E-ICL boosting LLM performance by over 4% on multiple datasets, even when the auxiliary model's performance is 10% lower than the LLM's.
  • Ablation studies confirm that all components of E-ICL (emotion-similar examples, dynamic soft labels, and exclusionary prediction) contribute to improved performance.
  • Case studies show E-ICL accurately predicting emotions like "caring" and "jealous" where standard ICL and Zero-Shot methods failed due to semantic similarity issues and difficulty distinguishing nuanced emotions.
  • ChatGPT-turbo demonstrates better robustness to noise introduced by E-ICL compared to Claude-haiku.

Limitations and Future Work

  • Limitations:
    • The paper acknowledges that ICL might conform to prototype theory across a broader range of tasks.
  • Semantically similar examples and fixed unique labels might not always adequately represent complex relationships.
  • While the exclusionary strategy is effective for multi-classification tasks, its applicability to other domains was not fully explored due to resource constraints.
  • Future Work:
    • Investigate if ICL conforms to prototype theory in more tasks beyond emotion recognition.
  • Explore better methods for constructing example-label pairs.
  • Study the applicability of the exclusionary prediction strategy in more diverse tasks.

Summary updated

Abstract

In-context learning (ICL) achieves remarkable performance in various domains such as knowledge acquisition, commonsense reasoning, and semantic understanding. However, its performance significantly deteriorates for emotion detection tasks, especially fine-grained emotion recognition. The underlying reasons for this remain unclear. In this paper, we identify the reasons behind ICL's poor performance from the perspective of prototype theory and propose a method to address this issue. Specifically, we conduct extensive pilot experiments and find that ICL conforms to the prototype theory on fine-grained emotion recognition. Based on this theory, we uncover the following deficiencies in ICL: (1) It relies on prototypes (example-label pairs) that are semantically similar but emotionally inaccurate to predict emotions. (2) It is prone to interference from irrelevant categories, affecting the accuracy and robustness of the predictions. To address these issues, we propose an Emotion Context Learning method (E-ICL) on fine-grained emotion recognition. E-ICL relies on more emotionally accurate prototypes to predict categories by referring to emotionally similar examples with dynamic labels. Simultaneously, E-ICL employs an exclusionary emotion prediction strategy to avoid interference from irrelevant categories, thereby increasing its accuracy and robustness. Note that the entire process is accomplished with the assistance of a plug-and-play emotion auxiliary model, without additional training. Experiments on the fine-grained emotion datasets EDOS, Empathetic-Dialogues, EmpatheticIntent, and GoEmotions show that E-ICL achieves superior emotion prediction performance. Furthermore, even when the emotion auxiliary model used is lower than 10% of the LLMs, E-ICL can still boost the performance of LLMs by over 4% on multiple datasets.
BibTeX
@misc{yang2024e,
  author = {Zhou Yang and Zhaochun Ren and Chenglong Ye and Yu‐Feng Wang and Haizhou Sun and Chao Chen and Xiaofei Zhu and Yunbing Wu and Xiangwen Liao},
  title = {{E-ICL: Enhancing Fine-Grained Emotion Recognition through the Lens of Prototype Theory}},
  year = {2024},
  eprint = {2406.02642},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2406.02642},
}
Figure