Personality-affected Emotion Generation in Dialog Systems

Zhiyuan Wen, Jiannong Cao, Jiaxing Shen, Ruosong Yang, Shuaiqi Liu, Maosong Sun

TL;DR

Most AI chatbots struggle with emotional consistency, often flipping between moods in ways that feel "robotic" or jarring. This paper introduces Personality-affected Emotion Generation, a framework that uses a speaker's personality traits to model how their mood shifts during a conversation, leading to significantly more human-like and consistent emotional responses.

What Problem Does This Paper Address?

While current dialog systems have become quite good at recognizing user emotions and attempting "empathy," they often lack a stable internal identity. Most models learn to mimic conversational patterns from anonymous data, which leads to a major issue: affective instability. A chatbot might be cheerful in one sentence and pessimistically empathetic the next, even if the user's attitude hasn't changed.

Psychological research tells us that human emotional expression is rooted in personality. For example, a highly neurotic person might react to stress with anger, while a more agreeable person might remain calm. This paper addresses the gap between "empathy" (reacting to the user) and "personality" (acting according to an internal trait), proposing a system that generates emotions based on a defined persona.

Methods

The authors propose a Personality-affected Mood Transition model. The core innovation is treating personality not just as a static "feature," but as a transition weight that dictates how a speaker’s mood evolves.

  1. The VAD Bridge: To make personality and emotion "speak the same language," the authors map everything into the Valence-Arousal-Dominance (VAD) space. This 3D coordinate system allows the model to mathematically relate personality traits (like the "Big Five" OCEAN model) to emotional states.
  2. Mood Transition Regression: Instead of jumping straight to an emotion, the model predicts a "mood state" (a region in VAD space). It calculates how the current dialog context shifts the speaker's mood, using the speaker's personality to weight that shift.
  3. Multi-Granularity Extraction: The model uses a fine-tuned BERT encoder combined with a specialized attention layer. This layer aligns three things: the semantic meaning of the text, word-level emotional embeddings, and utterance-level emotion labels.
  4. Joint Training: The system is trained to solve two tasks at once: predicting the next mood state (regression) and selecting the correct discrete emotion label (classification) using Focal Loss to handle the fact that "Neutral" is far more common than "Disgust" or "Fear."

Datasets

The authors introduced a new benchmark: PELD (Personality EmotionLines Dataset).

  • Source: It is built from the scripts of the TV show Friends, merging data from existing datasets like MELD and EmoryNLP.
  • Content: It contains 6,510 dialogue triples (Utterance 1, Utterance 2, and the Response) for the six main characters.
  • Annotations: Each triple includes emotion labels for every utterance and "Big Five" personality scores for the characters, providing a unique resource for studying how specific personas (like Chandler vs. Rachel) react differently to the same situation.

Key Insights and Ideas

  • Personality as a Search Constraint: In dialogue, many emotions could be "appropriate" for a single context. Personality acts as a filter that narrows down the search space, making the AI's choice feel more deliberate and "in character."
  • Mood vs. Emotion: The paper distinguishes between short-term emotions and mid-term mood states. By modeling the transition of the mood, the AI maintains a "memory" of its internal state, preventing the jarring emotional flips seen in simpler models.
  • Negative Emotion Sensitivity: One of the most interesting findings was that personality traits have a much stronger influence on how people transition out of negative emotions (like sadness or anger) than how they handle positive ones.
  • Beyond Concatenation: Simply "tacking on" a personality vector to a model (concatenation) provides unstable results. Modeling personality as the logic behind the mood shift is what actually drives performance.

Key Results

The proposed model was evaluated against several BERT-based baselines:

  • Accuracy Boost: The model improved emotion generation performance by 13% in macro-F1 and 5% in weighted-F1 over the standard BERT-base model.
  • Minority Emotions: The model was significantly better at generating rare emotions like "Disgust" (improving F1 from 0.012 to 0.167), which are usually ignored by models that over-optimize for the "Neutral" majority.
  • Consistency: Qualitative case studies showed that the model could successfully maintain a character's "vibe" (e.g., Rachel's high extroversion making her more susceptible to mood lifts from Michael's comfort).

Limitations and Future Work

  • Text-Only: The current model relies entirely on text. The authors acknowledge that in real human interaction, personality is expressed through voice, gestures, and facial expressions, which are not captured here.
  • Background Knowledge: The model sometimes fails when a response depends on deep "lore" or specific relationship history between characters that isn't present in the immediate 3-turn dialog window.
  • Label to Language: While the model is excellent at picking the right emotion label, the next step is integrating this into a full Natural Language Generation (NLG) pipeline to ensure the actual words spoken match the predicted emotion and personality perfectly.

Summary updated

Abstract

Generating appropriate emotions for responses is essential for dialogue systems to provide human-like interaction in various application scenarios. Most previous dialogue systems tried to achieve this goal by learning empathetic manners from anonymous conversational data. However, emotional responses generated by those methods may be inconsistent, which will decrease user engagement and service quality. Psychological findings suggest that the emotional expressions of humans are rooted in personality traits. Therefore, we propose a new task, Personality-affected Emotion Generation, to generate emotion based on the personality given to the dialogue system and further investigate a solution through the personality-affected mood transition. Specifically, we first construct a daily dialogue dataset, Personality EmotionLines Dataset (PELD), with emotion and personality annotations. Subsequently, we analyze the challenges in this task, i.e., (1) heterogeneously integrating personality and emotional factors and (2) extracting multi-granularity emotional information in the dialogue context. Finally, we propose to model the personality as the transition weight by simulating the mood transition process in the dialogue system and solve the challenges above. We conduct extensive experiments on PELD for evaluation. Results suggest that by adopting our method, the emotion generation performance is improved by 13% in macro-F1 and 5% in weighted-F1 from the BERT-base model.
BibTeX
@article{wen2024personality,
  author = {Zhiyuan Wen and Jiannong Cao and Jiaxing Shen and Ruosong Yang and Shuaiqi Liu and Maosong Sun},
  title = {{Personality-affected Emotion Generation in Dialog Systems}},
  journal = {ACM Trans. Information Systems},
  year = {2024},
  url = {https://openreview.net/forum?id=L9IUaJ0IJ6},
}

Loading related content...

Figure