Introduction: The Complexity of Conversational Emotion
Emotion Recognition in Conversations (ERC) is a challenging task that goes beyond simple sentiment analysis. Unlike isolated sentences, conversational utterances are deeply embedded in context. A speaker's emotion is influenced by their own previous state (self-dependency) and the reactions of others (inter-speaker dependency).
Current graph-based models often struggle because they treat all interactions uniformly. However, human conversations naturally fluctuate between two distinct patterns: monologues, where one person speaks at length, and dialogues, where participants engage in rapid back-and-forth exchanges.
As shown in Figure 1, monologues typically exhibit "emotional consistency," where the speaker's mood remains relatively stable. In contrast, dialogues often trigger "emotional transfers," where moods shift quickly based on the interaction. Existing models using fixed-size windows fail to account for these different "emotional velocities," often introducing noise or missing critical context.
The DIB-HGCN Framework
To address these challenges, researchers developed the Dynamic Interactive Bimodal Hypergraph Network (DIB-HGCN). This framework introduces three core innovations:
1. Bimodal Hypergraph Partitioning
Instead of a single, monolithic graph, DIB-HGCN constructs two specialized sub-hypergraphs:
- Monologic Sub-hypergraph: Groups consecutive utterances from the same speaker to capture stable emotional trends.
- Dialogic Sub-hypergraph: Groups utterances where speakers change, focusing on the immediate emotional impact of one person's words on another.
2. Adaptive Windowing and Node Selection
The model employs a dynamic approach to context. For monologues, it uses dynamic windows that adjust to the length of the speech, ensuring all relevant "consistent" information is captured. For dialogues, it uses fragment windows to isolate local interactions, preventing interference from distant, unrelated parts of the conversation.
3. Dual-Graph Architecture
The system utilizes a parallel structure: a hypergraph to capture high-order, multi-speaker relationships and a standard undirected graph for global, pairwise connections. A Hypergraph Attention Convolution mechanism then learns the relative importance of different utterances and contextual groups to refine emotional features.
Key Results and Performance
DIB-HGCN was evaluated on two major benchmarks: IEMOCAP (two-person theatrical sessions) and MELD (multi-party interactions from the TV show Friends).
The model achieved state-of-the-art (SOTA) performance:
- IEMOCAP: 72.58% Accuracy and 72.46% F1-score.
- MELD: 68.01% Accuracy and 66.61% F1-score.
Ablation studies confirmed that the "secret sauce" is the sub-hypergraph partitioning. Removing the distinction between monologue and dialogue led to a significant drop in performance, proving that interaction patterns are more critical than speaker identity alone.
Future Directions
While DIB-HGCN sets a new standard, challenges remain. The model still struggles with infrequent emotions (like "fear" or "disgust" in the MELD dataset) due to data imbalance. Additionally, the current framework relies on the availability of textual, acoustic, and visual data. Future work will focus on making the model robust to "incomplete data" scenarios, such as audio-only phone calls, to enhance its real-world applicability.
Summary updated