DetectiveNN: Imitating Human Emotional Reasoning with a Recall-Detect-Predict Framework for Emotion Recognition in Conversations

Simin Hong, Jun Sun, Taihao LiEMNLP 2024 · DOI

Introduction

Recognizing emotions in conversations (ERC) is a complex cognitive task. Unlike static sentiment analysis, ERC requires understanding how emotions evolve through historical context, speaker interactions, and the overall narrative flow. This process aligns with the Cognitive Theory of Constructed Emotion, which suggests that emotions are not just triggered but are constructed based on past experiences and current situational cues.

Existing ERC models often rely on a simple "recall-then-predict" approach, which captures dialogue history but lacks a dedicated mechanism to interpret emotional cues before classification. This often leads to confusion between nuanced emotional states, such as "angry" versus "frustrated." To address this, we present DetectiveNN, a framework that mimics human reasoning through a three-stage Recall-Detect-Predict process.

Proposed Framework

DetectiveNN structures the emotional reasoning process into three distinct phases, as illustrated in Figure 1.

The DetectiveNN architecture, illustrating the three-stage Recall-Detect-Predict process.

1. Recall Phase

The model utilizes bi-directional Gated Recurrent Units (GRUs) to gather two critical types of information:

  • Intra-context embeddings: Capturing the specific emotional journey of an individual speaker.
  • Global-context embeddings: Tracking the overall flow and sentiment of the conversation across all participants.

2. Detection Phase

This phase represents the core innovation of DetectiveNN. Instead of passing raw embeddings to a classifier, the model performs "detective work" using a Transformer-based architecture:

  • Examination: It analyzes speaker-specific cues to construct a coherent emotional narrative.
  • Cross-Verification: A cross-attention mechanism validates the speaker's internal narrative against the global conversation context. This step ensures the detected "vibe" is consistent with the external dialogue environment.

3. Prediction Phase

The final stage integrates the original context with the insights gained during the detection phase. A Multi-Layer Perceptron (MLP) and a Softmax layer are employed to predict the final emotional label for the utterance.

Experimental Setup

DetectiveNN was evaluated on three major ERC benchmarks:

  • IEMOCAP: A multimodal dataset featuring two-person scripted and improvised dialogues.
  • DailyDialog: A large-scale dataset of everyday human communication.
  • EmoryNLP: Transcripts from the TV show Friends, characterized by multi-party interactions and complex social dynamics.

Results and Discussion

DetectiveNN achieves state-of-the-art (SOTA) performance across all tested benchmarks. On the IEMOCAP dataset, the model reached an accuracy of 76.15%, a significant improvement over previous methods like EmoCaps.

Performance comparison on the IEMOCAP dataset, showing DetectiveNN achieving state-of-the-art results.

On DailyDialog, the model achieved 75.55% accuracy, outperforming DialogueCRN by 6.10%. The results on EmoryNLP also showed significant gains, reaching 42.68% accuracy.

Key Insights

  • Cognitive Grounding: Separating "interpreting cues" from "classifying labels" allows the model to handle emotional nuances more effectively than traditional end-to-end models.
  • Emotional Inertia: The framework successfully captures how a person's emotional state persists or evolves logically over several turns.
  • Disambiguation: Ablation studies confirmed that "cross-verification" is vital for distinguishing between closely related negative emotions. Without this step, the model's ability to differentiate between anger and frustration decreases significantly.

Conclusion

DetectiveNN demonstrates that mimicking human cognitive structures—specifically the act of interpreting cues against context—leads to superior performance in emotion recognition. While the model performs exceptionally well on long-context dialogues, it faces challenges with very short exchanges and lacks awareness of individual personality traits. Future work will explore integrating personality profiles to better handle complex social dynamics like sarcasm and humor.

Summary updated

BibTeX
@inproceedings{hong2024detectivenn,
  author = {Simin Hong and Jun Sun and Taihao Li},
  title = {{DetectiveNN: Imitating Human Emotional Reasoning with a Recall-Detect-Predict Framework for Emotion Recognition in Conversations}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2024},
  doi = {10.18653/v1/2024.findings-emnlp.536},
}
Figure