Human Emotion Knowledge Representation Emerges in Large Language Model and Supports Discrete Emotion Inference

Ming Li, Yusheng Su, Hsiu-Yuan Huang, Jiali Cheng, Xin Hu, Xinmiao Zhang, Huadong Wang, Yujia Qin, Xiaozhi Wang, Zhiyuan Liu, Dan ZhangarXiv 2023 · arXiv · OpenReview

Abstract

Understanding how language supports emotion inference remains a central debate in cognitive science. This study investigates whether language-derived emotion-concept knowledge causally supports emotion inference by manipulating language-specific knowledge representations within Large Language Models (LLMs). Using prompt-tuning techniques, we identified that 14 distinct attributes of emotion concepts are represented by specific artificial neuron populations. By manipulating these attribute-related neurons, we observed significant performance deterioration in emotion inference tasks compared to random manipulations. Furthermore, this deterioration correlated with the relative importance of these attributes in the human mental space. These findings provide causal evidence for a language-based mechanism of emotion inference and highlight the sophisticated role of emotion-concept knowledge in linguistic models.

Introduction

A fundamental question in emotion science is whether language is merely a tool for labeling pre-existing feelings or a foundational component that helps us categorize and understand emotional experiences. This debate typically pits the Embodied View—which suggests emotions are hard-wired categories tied to sensory-motor experiences—against the Constructivist View, which posits that conceptual knowledge and language shape our emotional perceptions.

Testing these theories in humans is challenging because it is impossible to isolate linguistic knowledge from a lifetime of sensory-motor experience. This study utilizes LLMs as a "pure language" testbed. Because LLMs lack biological bodies and sensory input, their ability to infer emotions provides a unique opportunity to determine if language-derived knowledge alone is sufficient for emotional intelligence.

Methodology

The researchers utilized RoBERTa as a representative LLM to explore how emotion is represented in high-dimensional linguistic space. The study followed a four-stage approach:

  1. Prompt Tuning: To elicit the model's maximum capability without altering its core weights, the team optimized 27 emotion-specific task prompts (sequences of pseudo-tokens). This allowed the model to perform emotion inference while its internal parameters remained frozen.
  2. Representational Similarity Analysis (RSA): The researchers compared the model's internal neuron activations to human ratings across 14 conceptual attributes, including core affect (valence, arousal), prototypical facial expressions, and situational appraisals (e.g., fairness, control).
  3. Causal Manipulation (Ablation): After identifying neurons associated with specific emotion attributes, the researchers systematically "silenced" these neurons (setting their activations to zero) to observe the impact on the model's inference performance.
  4. Human Benchmarking: Behavioral experiments were conducted to map the "human mental space" of emotions. Participants provided similarity judgments and attribute ratings, creating a ground-truth map to compare against the LLM's internal representations.

Data Sources

The study leveraged the GoEmotions dataset, which contains over 58,000 Reddit comments labeled across 27 emotion categories. Additionally, the researchers collected original human ratings to establish the importance of various conceptual attributes and the subjective similarity between different emotional states.

Key Findings

Emergent Experiential Knowledge

One of the most striking findings was that the LLM learned to represent "experiential" attributes—such as the specific facial muscle movements associated with disgust—solely through text. Despite never having seen a human face, the model's internal representations of these physical attributes correlated significantly with human ratings.

Distributed and Causal Representations

Emotion knowledge was not found to be localized in a single "emotion hub." Instead, the neurons responsible for representing emotion concepts were distributed across all layers of the network. Critically, the ablation experiments proved a causal link: when "emotion-concept" neurons were silenced, the model's ability to infer emotions dropped significantly more than when random neurons were removed. This suggests these neurons are functional components of the inference process, not just passive correlates.

Alignment with Human Mental Space

The study found a high degree of alignment between AI and human cognition. The attributes that humans consider most central to defining an emotion (such as valence or fairness) were the same attributes that, when ablated in the LLM, caused the most significant performance drops. Furthermore, the model's inference accuracy was highest for emotions where human raters also showed the highest agreement ($r = .797$).

The "Inverted-U" Effect

The researchers identified an "Inverted-U" relationship regarding the scale of manipulation. The strongest correlation between the model's internal knowledge and human mental space was observed when manipulating approximately 4,000 neurons. Manipulating too few neurons had negligible effects, while manipulating too many resulted in generalized degradation of the model's linguistic processing capabilities.

Discussion and Conclusion

These results provide robust support for the constructivist view, demonstrating that language-derived knowledge is sufficient to support complex emotion inference. While the study does not claim that LLMs "feel" emotions, it proves that they possess a sophisticated, human-like representation of emotion concepts.

For cognitive science, this suggests that the linguistic system provides a powerful framework for organizing emotional experience. Future research may focus on how these language-based representations integrate with sensory-motor systems in humans, using LLMs as a computational reference point for understanding the multi-modal nature of human affect.

Summary updated

Abstract

Understanding how language supports emotion inference remains a topic of debate in emotion science. The present study investigated whether language-derived emotion-concept knowledge would causally support emotion inference by manipulating the language-specific knowledge representations in large language models. Using the prompt technique, 14 attributes of emotion concepts were found to be represented by distinct artificial neuron populations. By manipulating these attribute-related neurons, the majority of the emotion inference tasks showed performance deterioration compared to random manipulations. The attribute-specific performance deterioration was related to the importance of different attributes in human mental space. Our findings provide causal evidence in support of a language-based mechanism for emotion inference and highlight the contributions of emotion-concept knowledge.
BibTeX
@misc{li2023human,
  author = {Ming Li and Yusheng Su and Hsiu-Yuan Huang and Jiali Cheng and Xin Hu and Xinmiao Zhang and Huadong Wang and Yujia Qin and Xiaozhi Wang and Zhiyuan Liu and Dan Zhang},
  title = {{Human Emotion Knowledge Representation Emerges in Large Language Model and Supports Discrete Emotion Inference}},
  year = {2023},
  eprint = {2302.09582},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2302.09582},
}
Figure