Abstract
Affective Computing (AC), integrating computer science, psychology, and cognitive science knowledge, aims to enable machines to recognize, interpret, and simulate human emotions. Affective Computing (AC) includes two mainstream tasks: Affective Understanding (AU) and Affective Generation (AG). Fine-tuning Pre-trained Language Models (PLMs) for AU tasks has succeeded considerably; however, these models lack generalization ability. The emergence of Large Language Models (LLMs), such as the ChatGPT series and LLaMA models, brings new opportunities and challenges, catalyzing a paradigm shift in AC. LLMs possess capabilities of in-context learning, common sense reasoning, and advanced sequence generation. This survey summarizes the development of LLM research in this field, outlining techniques like Instruction Tuning and Prompt Engineering, and summarizing existing benchmarks and evaluation methods.
Introduction
For decades, Affective Computing relied on "small" pre-trained models (like BERT or RoBERTa) that were fine-tuned on narrow, manually labeled datasets. While these models were effective for simple sentiment analysis, they were often "one-trick ponies" that struggled to generalize to new domains or generate nuanced, empathetic responses.
The rise of Large Language Models (LLMs) has fundamentally changed this landscape. As shown in Figure 1, the field is moving from task-specific datasets and models toward multi-task datasets and general-purpose LLMs that can be adapted through prompting and instruction tuning.
Technical Pillars and Taxonomy
The adaptation of LLMs for AC is organized into three primary technical pillars: Instruction Tuning, Prompt Engineering, and Reinforcement Learning. Figure 2 provides a comprehensive taxonomy of the current research landscape.
1. Instruction Tuning
This involves fine-tuning models on datasets where tasks are explicitly defined by natural language instructions. Researchers often utilize Parameter-Efficient Fine-Tuning (PEFT) methods like LoRA and Prompt-Tuning. These techniques allow models to learn emotional nuances and acquire "affect-aware" capabilities without the prohibitive cost of updating billions of parameters.
2. Prompt Engineering
Beyond simple zero-shot queries, advanced strategies have emerged to unlock the latent emotional intelligence of LLMs:
- Chain-of-Thought (CoT): Forcing the model to "think out loud" about emotional triggers before providing a label.
- Agent-based Prompting: Assigning the LLM a persona (e.g., "an empathetic therapist") or using multi-agent systems where different "experts" debate a user's emotional state.
3. Reinforcement Learning (RL)
RL is used to align models with human emotional preferences. This includes RLHF (Human Feedback), RLAIF (AI Feedback), and RLVR (Verifiable Rewards), where models are rewarded for generating responses that are polite, empathetic, and non-toxic.
Key Insights and Performance
- From Labeling to Reasoning: In the pre-LLM era, AC was primarily a classification problem. Now, it is a reasoning problem. Techniques like Three-hop Reasoning (THOR) allow models to identify an aspect, determine sentiment, and explain the underlying cause.
- The Empathy Gap: While LLMs like GPT-4 can sometimes exceed human performance in empathy levels (by roughly 10% in some studies), they still struggle with highly structured tasks like Aspect-Based Sentiment Analysis (ABSA) compared to smaller, specialized models.
- Instruction Tuning Efficacy: Applying LoRA or P-Tuning can improve performance across multiple affective datasets simultaneously, suggesting that emotional intelligence is a transferable skill within the model's weights.
Benchmarks and Evaluation
Evaluation has moved beyond traditional sentiment analysis staples like Yelp and IMDb. Newer benchmarks include:
- SOUL (Sentiment and Opinion Understanding of Language): Focuses on complex linguistic nuances.
- EQ-Bench: Specifically tests emotional intelligence in dialogue.
- Multimodal Sets: Benchmarks like MERBench and DFEW integrate video and audio cues alongside text, acknowledging that true emotional intelligence requires "seeing" and "hearing."
Limitations and Future Work
Despite the progress, several challenges remain. Current reward functions for RL often lack grounding in deep psychological theories, leading to "mechanical" empathy. Furthermore, capturing how emotions evolve over long, multi-turn conversations (temporal modeling) remains an open challenge. Future research must also address the ethical and privacy concerns inherent in processing deeply personal affective data.
Summary updated