Abstract: Emotion Recognition in Conversation (ERC) is an emerging subfield of sentiment analysis that shifts focus from isolated sentences to the dynamic, context-dependent nature of human dialogue. This paper provides a comprehensive overview of ERC, identifying why traditional NLP models struggle with conversational data. It highlights the critical roles of speaker-specific context and "emotional inertia" while benchmarking state-of-the-art deep learning models, such as DialogueRNN, against major datasets like IEMOCAP and MELD.
Introduction
Published in IEEE Access in 2019, this survey explores the burgeoning field of ERC. Unlike standard sentiment analysis, which often treats utterances as independent units, ERC recognizes that the meaning and emotional weight of a statement are heavily influenced by the preceding conversation and the specific speakers involved. The paper identifies key challenges, reviews modern deep learning architectures, and benchmarks performance across standard datasets.
Challenges in ERC
Traditional "vanilla" emotion recognition systems often fail in dialogue because they ignore several unique factors that define human interaction:
- Contextual Dependency: Emotions are influenced by both local context (the immediate previous turn) and distant context (the overall history of the talk).
- Emotional Inertia: Humans tend to stay in a specific emotional state unless an external stimulus triggers a shift.
- Inter-personal Dependencies: In a dialogue, one person’s emotional state often influences the other (e.g., one person’s dismissive comment causing the other’s anger).
- Speaker Specificity: Different people express emotions differently; a sarcastic person's "Great!" is very different from an enthusiast's "Great!"
Methodological Evolution
The paper reviews the transition from older lexicon-based approaches to modern deep learning architectures designed specifically for sequence and speaker modeling:
- Context Modeling: Using Recurrent Neural Networks (RNNs) and Attention mechanisms to "remember" previous turns in a conversation.
- DialogueRNN: A standout model that uses a hierarchical multi-stage RNN to track individual speaker states and inter-speaker influences.
- Memory Networks (CMN & ICON): These models use distinct memory banks for each speaker to model self-dependency and inter-personal dynamics.
- HRLCE (Hybrid RNN with Local Context Enrichment): A framework that combines traditional word embeddings (Glove) with contextualized ones (ELMo) and "DeepMoji" to capture nuances in short-text conversations.
Benchmarking Datasets
The authors compare several key benchmarks used to train and test these models:
- IEMOCAP: A popular multimodal dataset (video, audio, text) of scripted and improvised dyadic interactions.
- MELD: Based on the TV show Friends, this is a large-scale multiparty dataset that is more "noisy" and realistic than IEMOCAP.
- DailyDialog: A large, text-only dataset of human-written daily conversations.
- EmoContext: A dataset from a SemEval challenge focusing on very short, three-turn dialogues.
- SEMAINE: A dataset focused on real-valued affective attributes like valence (positivity) and arousal (intensity) rather than discrete labels.
Key Insights and Results
A major hurdle identified is the "Emotion Shift" Problem: Most current models are good at detecting stable emotions but fail when a speaker suddenly changes their mood. They tend to "mimic" the previous state too much due to the way RNNs process sequences. Additionally, the authors distinguish between recognizing an emotion and reasoning about its cause—an ideal AI should understand why a user is angry, not just that they are.
In benchmarking, DialogueRNN emerged as a top performer:
- On the IEMOCAP dataset, it achieved a weighted F1-score of 62.75%, significantly outperforming "vanilla" models like bc-LSTM (54.95%).
- On the SEMAINE dataset, it showed the lowest Mean Absolute Error (MAE) for valence and arousal, proving it is better at tracking emotional intensity over time.
- On the EmoContext benchmark, the HRLCE model led the pack with an F1-score of 76.66%, likely due to its use of multi-head attention and diverse embeddings.
Limitations and Future Work
The authors identify several frontiers that remain largely unconquered:
- Multiparty Scalability: Most models are designed for two people; scaling to five or six speakers (like in MELD) is computationally expensive and noisy.
- Fine-Grained Topics: Current models struggle to link emotions to specific topics (e.g., being happy about a promotion but sad about the commute).
- Lack of Causal Data: There is a lack of datasets annotated for "emotion reasoning"—the causal links between utterances.
- Real-time Constraints: Many high-performing models use "future" utterances to help classify "past" ones, which is impossible for real-time applications like medical chatbots.
Summary updated