Enhancing Cognitive Models of Emotions with Representation Learning

Yuting Guo, Jinho D. ChoiNAACL 2021

TL;DR

This paper introduces a novel deep learning framework that learns fine-grained emotion embeddings from text. By using a multi-head probing model, they not only achieve state-of-the-art emotion classification but also generate interpretable emotion graphs and wheels, bridging the gap between computational models and psychological theories of emotion.

What Problem Does This Paper Address?

Understanding and computationally representing human emotions is a long-standing challenge in psychology and AI. Existing approaches often fall into two camps: discrete theories that categorize emotions into basic types, and dimensional theories that represent emotions as points in a continuous space. While these models offer valuable insights, they often struggle with the nuances of fine-grained emotions and the subjective nature of emotional experience. Furthermore, deep learning models, while powerful for tasks like emotion classification, often act as "black boxes," making it difficult to understand how they learn and represent emotional concepts. This paper aims to bridge this gap by developing a method to learn interpretable emotion representations from text that can enhance existing cognitive models.

Methods

The core of the proposed method is a multi-head probing model. This model takes a document as input and first generates a contextualized document embedding using a BERT encoder. This embedding is then passed through multiple "probing heads," each designed to extract specific features useful for emotion classification. These probing heads are arranged in layers, allowing the model to learn hierarchical representations. Early layers might capture more abstract concepts (like valence or intensity), while later layers learn finer-grained emotional distinctions.

Crucially, the authors use probing to interpret these learned representations. By analyzing how different layers classify emotions, they can infer what kinds of emotional concepts are being captured at each stage. This allows them to derive an "emotion graph" that visualizes hierarchical relationships between emotions. They also leverage these embeddings to generate an "emotion wheel," similar to Plutchik's model, and to augment existing dimensional models like the PAD (Pleasure, Arousal, Dominance) model by predicting values for emotions not originally covered.

Datasets

The primary dataset used is the Empathetic Dialogue dataset. This dataset is notable for its annotation of 32 fine-grained emotions, which is significantly more than most existing emotion datasets that typically offer around 7 coarse-grained categories. The authors use the situation descriptions from this dataset as input for their models.

Key Insights and Ideas

  • Interpretable Emotion Embeddings: The framework moves beyond simply classifying emotions to understanding how emotions are represented computationally, making deep learning models more transparent in this domain.
  • Hierarchical Emotion Learning: The multi-head probing architecture effectively learns hierarchical representations, with lower layers capturing broader emotional concepts and higher layers refining them into more specific emotions.
  • Bridging Discrete and Dimensional Models: The learned embeddings can be used to generate both discrete structures (emotion graphs, wheels) and augment dimensional models (PAD values), offering a unified approach.
  • Augmenting Cognitive Models: The ability to predict PAD values for previously unannotated emotions suggests that these learned embeddings can enrich and extend existing psychological theories of emotion.

Key Results

  • State-of-the-Art Classification: The proposed model achieved state-of-the-art accuracy on the Empathetic Dialogue dataset for classifying 32 emotions.
  • Improved Baseline: Even a simple BERT model with no probing achieved a significantly higher accuracy (57.6%) than the previously reported state-of-the-art (48%).
  • Emotion Graph and Wheel Generation: The model successfully generated an emotion graph and an emotion wheel that showed meaningful relationships between emotions, often aligning with established psychological models. For instance, it found relations like "Excited = Anticipating + Joyful" and "Lonely = Sad + Afraid."
  • PAD Model Augmentation: The framework could predict PAD values for emotions missing from the original PAD model, and these predictions showed good consistency with existing data and human perception.

Limitations and Future Work

The paper acknowledges that the emotion wheel generation involves a manual selection of basic emotions from Plutchik's model. While the derived relationships are compatible, this initial step is not fully automated. Future work could explore more automated ways to identify these foundational emotions. Additionally, while the model demonstrates strong performance and interpretability, further research could explore its application to other emotion-related tasks or datasets, and investigate the impact of different encoder architectures or probing head designs. The authors also suggest that these emotion representations could be used to generate more nuanced and contextually appropriate emotional responses in dialogue systems.

Summary updated

BibTeX
@inproceedings{guo2021enhancing,
  author = {Yuting Guo and Jinho D. Choi},
  title = {{Enhancing Cognitive Models of Emotions with Representation Learning}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2021},
}
Figure