Abstract: The capacity for empathy is crucial to the success of open-domain dialog systems. Due to its multi-dimensional nature, various factors relate to empathy expression, such as communication mechanisms, dialog acts, and emotion. However, existing methods for empathetic response generation usually either consider only one empathy factor or ignore the hierarchical relationships between different factors, leading to a weak ability in empathy modeling. In this paper, we propose a multi-factor hierarchical framework, CoMAE, for empathetic response generation, which models the above three key factors of empathy expression in a hierarchical way. We show experimentally that our CoMAE-based model can generate more empathetic responses than previous methods. We also highlight the importance of hierarchical modeling of different factors through both empirical analysis on a real-life corpus and extensive experiments.
Introduction
Building a chatbot that truly "understands" and responds with empathy is a significant challenge in Natural Language Processing. Most existing models treat empathy as a one-dimensional problem, usually focusing solely on recognizing the user's emotion. However, human empathy is inherently multi-dimensional, involving a cognitive aspect (understanding the situation) and an affective aspect (reacting with appropriate emotion).
Recent research has identified three key factors in expressing empathy: Communication Mechanisms (CM) (high-level strategies like emotional reaction or exploration), Dialog Acts (DA) (specific intents like questioning or consoling), and Emotions (EM) (the affective state, such as admiration or sadness). The gap identified in current research is that previous models either ignore these factors or treat them as independent variables. In reality, these factors are deeply interdependent: a specific communication strategy naturally leads to certain dialog acts and emotional expressions.
The CoMAE Framework
The authors propose CoMAE (Communication mechanism, dialog Act, and Emotion), a hierarchical framework built on top of the GPT-2 architecture. The core innovation is the modeling of hierarchical dependency between the three factors.
As shown in the architecture, the model follows a sequential prediction process:
- Predict CM: The model first predicts the high-level Communication Mechanism based on the conversation context.
- Predict DA: It then predicts the Dialog Act, conditioned on both the context and the previously predicted CM.
- Predict EM: Finally, it predicts the Emotion, conditioned on the context, the CM, and the DA.
These three factors are fused into a single "CoMAE embedding," which is added to the input token embeddings to guide the GPT-2 decoder. The model is trained using a multi-task objective that combines standard language modeling loss with prediction losses for the three empathy factors.
Experimental Setup
The researchers utilized a large-scale empathetic conversation corpus crawled from Reddit, specifically from the "Happy" (positive) and "Offmychest" (negative) subreddits. To handle the lack of labels in raw data, the authors employed specialized resources for automatic annotation: CM was annotated using a classifier trained on TalkLife data; DA was annotated using a taxonomy of 8 high-frequency intents; and EM was annotated using a modified GoEmotions taxonomy.
Empirical analysis of these annotations confirmed the hierarchical assumption. For example, heat maps of conditional distributions revealed that choosing the "Exploration" mechanism almost always correlates with a "Questioning" dialog act and expressions of "Surprise."
Results and Discussion
The CoMAE framework was tested against several baselines, including vanilla GPT-2 and "flat" models that predicted factors independently (denoted as CM || DA || EM).
As seen in the results table, CoMAE (CM → DA → EM) achieved the best performance across all metrics, including lower Perplexity (PPL) and higher BLEU (B-2) and ROUGE (R-L) scores. The hierarchical version consistently outperformed the independent version, suggesting that conditioning reduces noise and prevents "conflicting" signals (e.g., a model trying to be "Consoling" while expressing "Anger").
Manual evaluation further supported these findings, with CoMAE-generated responses rated significantly higher for empathy and coherence. Additionally, "realization scores"—which measure how well the generated text matches the intended factors—were significantly improved by the hierarchical approach.
Conclusion and Future Work
CoMAE demonstrates that treating empathy as a structured, hierarchical process rather than a set of independent variables produces more coherent and human-like responses. While the current framework applies embeddings uniformly to every token, future work may explore more granular control for multi-sentence responses and the incorporation of long-term persona-based planning to handle complex, multi-turn empathetic support.
Summary updated