Introduction
Recognizing emotions in conversations (ERC) is a complex cognitive task. Unlike static sentiment analysis, ERC requires understanding how emotions evolve through historical context, speaker interactions, and the overall narrative flow. This process aligns with the Cognitive Theory of Constructed Emotion, which suggests that emotions are not just triggered but are constructed based on past experiences and current situational cues.
Existing ERC models often rely on a simple "recall-then-predict" approach, which captures dialogue history but lacks a dedicated mechanism to interpret emotional cues before classification. This often leads to confusion between nuanced emotional states, such as "angry" versus "frustrated." To address this, we present DetectiveNN, a framework that mimics human reasoning through a three-stage Recall-Detect-Predict process.
Proposed Framework
DetectiveNN structures the emotional reasoning process into three distinct phases, as illustrated in Figure 1.
1. Recall Phase
The model utilizes bi-directional Gated Recurrent Units (GRUs) to gather two critical types of information:
- Intra-context embeddings: Capturing the specific emotional journey of an individual speaker.
- Global-context embeddings: Tracking the overall flow and sentiment of the conversation across all participants.
2. Detection Phase
This phase represents the core innovation of DetectiveNN. Instead of passing raw embeddings to a classifier, the model performs "detective work" using a Transformer-based architecture:
- Examination: It analyzes speaker-specific cues to construct a coherent emotional narrative.
- Cross-Verification: A cross-attention mechanism validates the speaker's internal narrative against the global conversation context. This step ensures the detected "vibe" is consistent with the external dialogue environment.
3. Prediction Phase
The final stage integrates the original context with the insights gained during the detection phase. A Multi-Layer Perceptron (MLP) and a Softmax layer are employed to predict the final emotional label for the utterance.
Experimental Setup
DetectiveNN was evaluated on three major ERC benchmarks:
- IEMOCAP: A multimodal dataset featuring two-person scripted and improvised dialogues.
- DailyDialog: A large-scale dataset of everyday human communication.
- EmoryNLP: Transcripts from the TV show Friends, characterized by multi-party interactions and complex social dynamics.
Results and Discussion
DetectiveNN achieves state-of-the-art (SOTA) performance across all tested benchmarks. On the IEMOCAP dataset, the model reached an accuracy of 76.15%, a significant improvement over previous methods like EmoCaps.
On DailyDialog, the model achieved 75.55% accuracy, outperforming DialogueCRN by 6.10%. The results on EmoryNLP also showed significant gains, reaching 42.68% accuracy.
Key Insights
- Cognitive Grounding: Separating "interpreting cues" from "classifying labels" allows the model to handle emotional nuances more effectively than traditional end-to-end models.
- Emotional Inertia: The framework successfully captures how a person's emotional state persists or evolves logically over several turns.
- Disambiguation: Ablation studies confirmed that "cross-verification" is vital for distinguishing between closely related negative emotions. Without this step, the model's ability to differentiate between anger and frustration decreases significantly.
Conclusion
DetectiveNN demonstrates that mimicking human cognitive structures—specifically the act of interpreting cues against context—leads to superior performance in emotion recognition. While the model performs exceptionally well on long-context dialogues, it faces challenges with very short exchanges and lacks awareness of individual personality traits. Future work will explore integrating personality profiles to better handle complex social dynamics like sarcasm and humor.
Summary updated