DialogueCRN: Contextual Reasoning Networks for Emotion Recognition in Conversations

Dou Hu, Lingwei Wei, Xiaoyong Huai

TL;DR

DialogueCRN introduces a cognitive-inspired approach to Emotion Recognition in Conversations (ERC) by mimicking the human dual-process theory of thought. By alternating between intuitive context retrieval and conscious logical reasoning, the model effectively extracts and orders emotional clues, setting new performance benchmarks across multiple conversation datasets.

What Problem Does This Paper Address?

To build empathetic conversational agents, machines must understand the emotional state of a speaker at any given moment. While deep learning models have gotten better at tracking who is speaking (speaker-level context) and the overall flow of the conversation (situation-level context), they struggle to truly comprehend the underlying emotional triggers.

The authors identify two main challenges in existing Emotion Recognition in Conversations (ERC) models:

  1. Clue Extraction: Most models retrieve context from static memories, missing rich, implicit emotional cues.
  2. Clue Integration: Models heavily rely on standard attention mechanisms, which treat contextual clues like an unordered bag of words. This ignores the intrinsic semantic and logical order of a conversation (e.g., a cause leading to an emotional effect), making it difficult to pinpoint what actually triggered an emotion.

Methods

Inspired by the Cognitive Theory of Emotion, DialogueCRN splits the task into two main phases: a Perception Phase and a Cognition Phase.

Figure 1: The architecture of the proposed model DialogueCRN.

1. Perception Phase

The model uses two separate bi-directional LSTMs to extract raw contextual features from text utterances:

  • Situation-level LSTM: Captures sequential dependencies between adjacent utterances in the conversation.
  • Speaker-level LSTM: Tracks the self-dependencies and internal states of individual speakers. The outputs are projected into linear layers to form Global Memories ($G^s$ and $G^v$).

2. Cognition Phase

Instead of using a simple attention layer over these memories, DialogueCRN introduces a multi-turn Reasoning Module that mimics human unique cognitive thinking through two alternating processes:

  • Conscious Reasoning Process: An LSTM updates a dynamic Working Memory ($h^{(t)}$). This step is slower but rational, helping the model maintain the logical and sequential order of clues.
  • Intuitive Retrieving Process: An attention mechanism, guided by the current working memory, queries the Global Memory to pull out the most relevant emotional clues.

This loop repeats for $T$ turns, iteratively refining the model's understanding before passing the final integrated clues to a Softmax classifier for emotion prediction.

Figure 2: The detailed structure of reasoning module.

Datasets

The authors evaluated DialogueCRN on three widely used public benchmark datasets:

  • IEMOCAP: Two-way conversations annotated with six discrete emotions (happy, sad, neutral, angry, excited, frustrated).
  • SEMAINE: Human-agent interactions annotated with four continuous affective attributes (Arousal, Expectancy, Power, Valence).
  • MELD: Multi-party conversations from the TV show Friends, annotated with seven emotion labels.

Note: While these are inherently multimodal datasets (containing text, audio, and video), this paper focuses strictly on emotion recognition from textual transcripts.

Key Insights and Ideas

  • Logical Order is Crucial for Emotion: Standard attention mechanisms lose the logical relationships between conversational events. By using an LSTM as a "working memory" during the reasoning phase, DialogueCRN preserves the chronological and causal flow of the dialogue.
  • Perception vs. Cognition Split: Ablation studies revealed that speaker-level context is highly effective during the intuitive perception phase (tracking who is talking). Conversely, complex situation-level context (the actual narrative or conflict) benefits far more from multi-turn cognitive reasoning.
  • Cognitive Modeling Works: Simulating a human-like "think, retrieve, rethink" loop directly translates to better feature representations for NLP classification tasks.

Key Results

DialogueCRN consistently outperformed strong baselines (including DialogueRNN and DialogueGCN) across all datasets:

  • IEMOCAP: Achieved a 66.05% Accuracy and 66.20% Weighted-F1 score, representing a $3.2\%$ and $4.0\%$ relative improvement over the previous best-performing baseline.
  • SEMAINE: Significantly reduced Mean Absolute Error (MAE), notably achieving an 11.1% improvement in tracking Arousal.
  • MELD: Improved Weighted-F1 by 2.9%, proving effective even in shorter conversations with high speaker turn-over.
Figure 4: The case study.

A case study from IEMOCAP (Figure 4) highlights the model's strength: when a female speaker's expectation for financial compensation goes unmet, DialogueCRN successfully traces the chain of context (8 $\rightarrow$ 7 $\rightarrow$ 2 $\rightarrow$ 1) to correctly classify her final state as frustrated/depressed, whereas baseline models misclassified it as angry or neutral.

Limitations and Future Work

The primary limitation acknowledged by the authors is that DialogueCRN currently operates only on textual features. Because the underlying datasets (IEMOCAP, MELD, SEMAINE) contain rich acoustic and visual data, extending this cognitive reasoning framework to a multimodal architecture is the immediate next step for future work.

Summary updated

Abstract

Emotion Recognition in Conversations (ERC) has gained increasing attention for developing empathetic machines. Recently, many approaches have been devoted to perceiving conversational context by deep learning models. However, these approaches are insufficient in understanding the context due to lacking the ability to extract and integrate emotional clues. In this work, we propose novel Contextual Reasoning Networks (DialogueCRN) to fully understand the conversational context from a cognitive perspective. Inspired by the Cognitive Theory of Emotion, we design multi-turn reasoning modules to extract and integrate emotional clues. The reasoning module iteratively performs an intuitive retrieving process and a conscious reasoning process, which imitates human unique cognitive thinking. Extensive experiments on three public benchmark datasets demonstrate the effectiveness and superiority of the proposed model.
BibTeX
@inproceedings{hu2021dialoguecrn,
  author = {Dou Hu and Lingwei Wei and Xiaoyong Huai},
  title = {{DialogueCRN: Contextual Reasoning Networks for Emotion Recognition in Conversations}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2021},
}

Loading related content...

Figure