TL;DR
DialogueCRN introduces a cognitive-inspired approach to Emotion Recognition in Conversations (ERC) by mimicking the human dual-process theory of thought. By alternating between intuitive context retrieval and conscious logical reasoning, the model effectively extracts and orders emotional clues, setting new performance benchmarks across multiple conversation datasets.
What Problem Does This Paper Address?
To build empathetic conversational agents, machines must understand the emotional state of a speaker at any given moment. While deep learning models have gotten better at tracking who is speaking (speaker-level context) and the overall flow of the conversation (situation-level context), they struggle to truly comprehend the underlying emotional triggers.
The authors identify two main challenges in existing Emotion Recognition in Conversations (ERC) models:
- Clue Extraction: Most models retrieve context from static memories, missing rich, implicit emotional cues.
- Clue Integration: Models heavily rely on standard attention mechanisms, which treat contextual clues like an unordered bag of words. This ignores the intrinsic semantic and logical order of a conversation (e.g., a cause leading to an emotional effect), making it difficult to pinpoint what actually triggered an emotion.
Methods
Inspired by the Cognitive Theory of Emotion, DialogueCRN splits the task into two main phases: a Perception Phase and a Cognition Phase.
1. Perception Phase
The model uses two separate bi-directional LSTMs to extract raw contextual features from text utterances:
- Situation-level LSTM: Captures sequential dependencies between adjacent utterances in the conversation.
- Speaker-level LSTM: Tracks the self-dependencies and internal states of individual speakers. The outputs are projected into linear layers to form Global Memories ($G^s$ and $G^v$).
2. Cognition Phase
Instead of using a simple attention layer over these memories, DialogueCRN introduces a multi-turn Reasoning Module that mimics human unique cognitive thinking through two alternating processes:
- Conscious Reasoning Process: An LSTM updates a dynamic Working Memory ($h^{(t)}$). This step is slower but rational, helping the model maintain the logical and sequential order of clues.
- Intuitive Retrieving Process: An attention mechanism, guided by the current working memory, queries the Global Memory to pull out the most relevant emotional clues.
This loop repeats for $T$ turns, iteratively refining the model's understanding before passing the final integrated clues to a Softmax classifier for emotion prediction.
Datasets
The authors evaluated DialogueCRN on three widely used public benchmark datasets:
- IEMOCAP: Two-way conversations annotated with six discrete emotions (happy, sad, neutral, angry, excited, frustrated).
- SEMAINE: Human-agent interactions annotated with four continuous affective attributes (Arousal, Expectancy, Power, Valence).
- MELD: Multi-party conversations from the TV show Friends, annotated with seven emotion labels.
Note: While these are inherently multimodal datasets (containing text, audio, and video), this paper focuses strictly on emotion recognition from textual transcripts.
Key Insights and Ideas
- Logical Order is Crucial for Emotion: Standard attention mechanisms lose the logical relationships between conversational events. By using an LSTM as a "working memory" during the reasoning phase, DialogueCRN preserves the chronological and causal flow of the dialogue.
- Perception vs. Cognition Split: Ablation studies revealed that speaker-level context is highly effective during the intuitive perception phase (tracking who is talking). Conversely, complex situation-level context (the actual narrative or conflict) benefits far more from multi-turn cognitive reasoning.
- Cognitive Modeling Works: Simulating a human-like "think, retrieve, rethink" loop directly translates to better feature representations for NLP classification tasks.
Key Results
DialogueCRN consistently outperformed strong baselines (including DialogueRNN and DialogueGCN) across all datasets:
- IEMOCAP: Achieved a 66.05% Accuracy and 66.20% Weighted-F1 score, representing a $3.2\%$ and $4.0\%$ relative improvement over the previous best-performing baseline.
- SEMAINE: Significantly reduced Mean Absolute Error (MAE), notably achieving an 11.1% improvement in tracking Arousal.
- MELD: Improved Weighted-F1 by 2.9%, proving effective even in shorter conversations with high speaker turn-over.
A case study from IEMOCAP (Figure 4) highlights the model's strength: when a female speaker's expectation for financial compensation goes unmet, DialogueCRN successfully traces the chain of context (8 $\rightarrow$ 7 $\rightarrow$ 2 $\rightarrow$ 1) to correctly classify her final state as frustrated/depressed, whereas baseline models misclassified it as angry or neutral.
Limitations and Future Work
The primary limitation acknowledged by the authors is that DialogueCRN currently operates only on textual features. Because the underlying datasets (IEMOCAP, MELD, SEMAINE) contain rich acoustic and visual data, extending this cognitive reasoning framework to a multimodal architecture is the immediate next step for future work.
Summary updated