Fine-grained Affective Processing Capabilities Emerging from Large Language Models

Joost Broekens, Bernhard Hilpert, Suzan Verberne, Kim Baraka, Patrick Gebhard, Aske Plaat

Introduction

Presented at the 2023 Conference on Affective Computing and Intelligent Interaction (ACII), this research investigates a fundamental question in artificial intelligence: does the massive scale of Large Language Models (LLMs) grant them an "emergent" understanding of human emotion? By evaluating ChatGPT (GPT-3.5) on established affective computing benchmarks, the study demonstrates that these models can perform sophisticated sentiment analysis and adhere to complex psychological appraisal rules in a zero-shot capacity, often rivaling specialized, fine-tuned models.

The Challenge of Context in Affective Computing

Traditionally, the field of Affective Computing—the study of systems that recognize, interpret, and simulate human affect—has followed two primary paths:

  1. Rule-based Symbolic AI: Hard-coding psychological theories, such as the OCC model, into logical frameworks.
  2. Specialized Machine Learning: Training models on specific, labeled datasets for emotion detection.

A persistent hurdle for both approaches is the role of context. Understanding an emotion requires a nuanced grasp of the "who, what, and where" of a situation. This research explores whether LLMs, which are pre-trained on vast amounts of human discourse, can bridge this gap without explicit emotional training. Specifically, the authors test the model's grasp of the Valence-Arousal-Dominance (VAD) framework and the Ortony-Clore-Collins (OCC) appraisal model, which describes how cognitive evaluations of events lead to specific emotional states.

Methodology

The researchers employed instruction prompting (zero-shot) to evaluate ChatGPT-3.5 across three Research Questions (RQs):

  • RQ1 (Sentiment Analysis): Can the model assign accurate VAD scores (0.0 to 1.0) to situational descriptions and emotion words?
  • RQ2 (Affect Representation): Can the model map situations to specific emotion words using both numerical Euclidean distance (based on VAD scores) and latent semantic matching?
  • RQ3 (Emotion Elicitation): Can the model act as an inference engine for the OCC model? The researchers "programmed" the logic-based OCC framework directly into a prompt to see if the model could predict emotions based on appraisal rules.

Datasets

The study utilized three primary benchmarks:

  • ANET (Affective Norms for English Text): 120 situational descriptions with human-validated VAD ratings.
  • Russell & Mehrabian Word List: 151 emotion words with associated ground-truth VAD scores.
  • Custom Appraisal Scenarios: 12 scenarios designed to test specific branches of the OCC model (e.g., comparing reactions to success vs. failure).

Key Insights

Emergent Intelligence and Perspective

The model’s affective capabilities appear to be a byproduct of learning complex language patterns rather than a dedicated module. A critical finding involved the "Dominance" dimension (the feeling of being in control). While the model initially struggled, its accuracy improved significantly when the prompt clarified the perspective (e.g., "assess the control the main person feels"). This suggests that LLMs are highly sensitive to the subject of an emotional context.

Latent vs. Numerical Processing

ChatGPT demonstrated a superior ability to "feel" the connection between a situation and an emotion word through latent semantic representation rather than through explicit calculation. When forced to use its own numerical VAD scores to match words, performance was lower than when it was allowed to match them based on meaning alone.

Programmable Psychology

The success of the OCC experiment indicates that LLMs can serve as an "inference engine" for psychological theories. By "soft-coding" a theory of mind into a prompt, researchers can leverage the model's reasoning capabilities to simulate complex emotional appraisals.

Results and Performance

The model's performance on the 120 ANET situations showed remarkably strong Pearson correlations with human ground truth:

  • Valence: $\rho = 0.95$
  • Arousal: $\rho = 0.82$
  • Dominance: $\rho = 0.82$

These results are comparable to, or exceed, state-of-the-art fine-tuned models like XLM-RoBERTa. Furthermore, using the prompted OCC rules, ChatGPT correctly predicted the emotion in 10 out of 12 complex scenarios, only struggling with the subtle distinction between "distress" and "disappointment."

Limitations and Future Directions

While the results are promising, the authors note several limitations. The study was conducted on GPT-3.5, and the rapid evolution of models like GPT-4 may yield different results. The model also exhibits a "semantic bias," occasionally prioritizing words that sound appropriate over those that are numerically accurate in VAD space.

Finally, the study focused on "snapshots" of emotion. Future research is required to determine if LLMs can simulate emotional dynamics over time—such as mood decay or transitions between emotional states—and whether these capabilities remain consistent across diverse, multi-modal, and cross-cultural datasets.

Summary updated

Abstract

Large language models, in particular generative pre-trained transformers (GPTs), show impressive results on a wide variety of language-related tasks. In this paper, we explore ChatGPT’s zero-shot ability to perform affective computing tasks using prompting alone. We show that ChatGPT a) performs meaningful sentiment analysis in the Valence, Arousal and Dominance dimensions, b) has meaningful emotion representations in terms of emotion categories and these affective dimensions, and c) can perform basic appraisal-based emotion elicitation of situations based on a prompt-based computational implementation of the OCC appraisal model. These findings are highly relevant: First, they show that the ability to solve complex affect processing tasks emerges from language-based token prediction trained on extensive data sets. Second, they show the potential of large language models for simulating, processing and analyzing human emotions, which has important implications for various applications such as sentiment analysis, socially interactive agents, and social robotics.
BibTeX
@inproceedings{broekens2023fine,
  author = {Joost Broekens and Bernhard Hilpert and Suzan Verberne and Kim Baraka and Patrick Gebhard and Aske Plaat},
  title = {{Fine-grained Affective Processing Capabilities Emerging from Large Language Models}},
  booktitle = {ACII},
  year = {2023},
  url = {https://openreview.net/forum?id=xrWuITKWJN},
}
Figure