Dynamic Interactive Bimodal Hypergraph Networks for Emotion Recognition in Conversations

Xuping Chen, Wuzhen ShiAAAI 2025 · DOI

Introduction: The Complexity of Conversational Emotion

Emotion Recognition in Conversations (ERC) is a challenging task that goes beyond simple sentiment analysis. Unlike isolated sentences, conversational utterances are deeply embedded in context. A speaker's emotion is influenced by their own previous state (self-dependency) and the reactions of others (inter-speaker dependency).

Current graph-based models often struggle because they treat all interactions uniformly. However, human conversations naturally fluctuate between two distinct patterns: monologues, where one person speaks at length, and dialogues, where participants engage in rapid back-and-forth exchanges.

Distinguishing between monologue and dialogue interactions in a conversation.

As shown in Figure 1, monologues typically exhibit "emotional consistency," where the speaker's mood remains relatively stable. In contrast, dialogues often trigger "emotional transfers," where moods shift quickly based on the interaction. Existing models using fixed-size windows fail to account for these different "emotional velocities," often introducing noise or missing critical context.

The DIB-HGCN Framework

To address these challenges, researchers developed the Dynamic Interactive Bimodal Hypergraph Network (DIB-HGCN). This framework introduces three core innovations:

1. Bimodal Hypergraph Partitioning

Instead of a single, monolithic graph, DIB-HGCN constructs two specialized sub-hypergraphs:

  • Monologic Sub-hypergraph: Groups consecutive utterances from the same speaker to capture stable emotional trends.
  • Dialogic Sub-hypergraph: Groups utterances where speakers change, focusing on the immediate emotional impact of one person's words on another.

2. Adaptive Windowing and Node Selection

The model employs a dynamic approach to context. For monologues, it uses dynamic windows that adjust to the length of the speech, ensuring all relevant "consistent" information is captured. For dialogues, it uses fragment windows to isolate local interactions, preventing interference from distant, unrelated parts of the conversation.

The node selection process for constructing sub-hypergraphs based on interaction patterns.

3. Dual-Graph Architecture

The system utilizes a parallel structure: a hypergraph to capture high-order, multi-speaker relationships and a standard undirected graph for global, pairwise connections. A Hypergraph Attention Convolution mechanism then learns the relative importance of different utterances and contextual groups to refine emotional features.

Key Results and Performance

DIB-HGCN was evaluated on two major benchmarks: IEMOCAP (two-person theatrical sessions) and MELD (multi-party interactions from the TV show Friends).

The model achieved state-of-the-art (SOTA) performance:

  • IEMOCAP: 72.58% Accuracy and 72.46% F1-score.
  • MELD: 68.01% Accuracy and 66.61% F1-score.

Ablation studies confirmed that the "secret sauce" is the sub-hypergraph partitioning. Removing the distinction between monologue and dialogue led to a significant drop in performance, proving that interaction patterns are more critical than speaker identity alone.

Future Directions

While DIB-HGCN sets a new standard, challenges remain. The model still struggles with infrequent emotions (like "fear" or "disgust" in the MELD dataset) due to data imbalance. Additionally, the current framework relies on the availability of textual, acoustic, and visual data. Future work will focus on making the model robust to "incomplete data" scenarios, such as audio-only phone calls, to enhance its real-world applicability.

Summary updated

BibTeX
@inproceedings{chen2025dynamic,
  author = {Xuping Chen and Wuzhen Shi},
  title = {{Dynamic Interactive Bimodal Hypergraph Networks for Emotion Recognition in Conversations}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i2.32114},
}
Figure