Introduction
Human communication is inherently multi-faceted, often blending various emotions with nuanced intensities. However, most current conversational AI models are limited to single-emotion generation at fixed levels, resulting in responses that feel robotic or emotionally flat. This paper introduces a novel framework designed to generate responses that can express multiple emotions with fine-grained intensity control, moving closer to the complexity of real human interaction.
The Problem: The "Bottleneck of Expression"
Existing dialogue systems face a "bottleneck of expression" due to two primary factors:
- Data Scarcity: There is a lack of large-scale datasets that provide labels for both multiple emotions and their specific intensities within a single utterance.
- Architectural Gaps: Current models lack internal mechanisms to track the "expenditure" of emotional content during sentence generation, often failing to maintain the target intensity or balance multiple emotional goals.
The MEI-DG Framework
The authors propose the Multiple Emotion with Intensity-based Dialogue Generation (MEI-DG) framework. Built on a Hierarchical Encoder-Decoder (HRED) backbone, it introduces a dual-memory architecture to manage emotional dynamics.
Dual-Memory Architecture
- Implicit Memory: This module acts as a dynamic emotional "gas tank." Using Valence-Arousal-Dominance (VAD) embeddings, it calculates how much of a target emotion has been expressed as words are generated. This ensures the model sustains the desired emotional tone until the sentence is complete.
- Explicit Memory: This component serves as a gatekeeper, deciding at each time step whether to select a generic functional word or an emotionally charged lexical item.
Intensity Control via Gaussian Kernels
To handle varying degrees of emotional strength, the model employs a Gaussian Kernel Layer. This allows the system to map vocabulary to an intensity space, enabling it to choose words that specifically match a user-defined intensity level—for instance, selecting "thrilled" instead of "happy" for high-intensity joy.
The MEIMD Dataset
To train and evaluate this framework, the researchers developed the MEIMD (Multiple Emotion Intensity aware Multi-party Dialogue) dataset. It consists of approximately 34,000 conversations extracted from eight popular TV series, including Friends, Breaking Bad, and Game of Thrones. Unlike previous benchmarks, each utterance is annotated with multiple emotions (from a set of seven primary emotions plus "Neutral") and an intensity score ranging from 0 to 3.
Experimental Results
The MEI-DG model was evaluated against several strong baselines, including ECM and Affect-LM, using both automatic metrics and human judgment.
As shown in the results, MEI-DG achieved the lowest Perplexity (71.2), indicating higher fluency and predictability. It also outperformed baselines in embedding-based metrics, suggesting that its responses are more semantically relevant to the conversation context. In human evaluations, MEI-DG was rated significantly higher for emotion consistency (43.9%) and intensity accuracy (37.2%) compared to standard hierarchical models.
Conclusion and Future Work
By treating emotion as a dynamic state that fluctuates word-by-word rather than a static label, MEI-DG represents a significant step toward more human-like conversational agents. While the model currently faces challenges with rare emotion pairings (e.g., simultaneous joy and anger), future work may involve integrating multi-modal cues—such as visual or acoustic data from the original TV show sources—to further refine emotional context and generation accuracy.
Summary updated