Do Language Models Feel the Sentiment? Investigating Emotion Vectors

Ali Hassan Khan, Saad Bin Ather, Muhammad Saif, Maryam Bashir, Hajra WaheedarXiv 2026 · arXiv

TL;DR

This paper investigates whether "emotion vectors"—specific directions in a language model's internal representation—are the primary drivers behind sentiment classification. By analyzing Qwen2.5 models, the researchers found that while these vectors contain significant sentiment information and causally influence model behavior, they do not necessarily represent a unique or superior mechanism compared to simpler confidence-based baselines.

What Problem Does This Paper Address?

Large Language Models (LLMs) are remarkably good at sentiment analysis, but the internal mechanics of how they arrive at these labels remain opaque. We know LLMs encode emotional concepts (like "joy" or "anger"), but does the model actually use these emotional representations to decide if a tweet is positive or negative?

The authors aim to disentangle three distinct concepts:

  1. Accessible Information: Can we find sentiment info hidden in emotion-related activations?
  2. Causal Influence: Does changing these activations actually change the model's output?
  3. Predictive Utility: Does knowing these emotional features help us build better sentiment classifiers than just looking at the model's final output?

Methods

The study focuses on two models: Qwen2.5-0.5B-Instruct and Qwen2.5-3B-Instruct. The workflow is summarized in Figure 1.

Figure 1: Overview of the study. Emotion directions from GoEmotions are used for representation analysis, interventions, and sentiment correction. The correction classifiers are also tested on Amazon without retraining.

Key methodological steps included:

  • Vector Extraction: The authors extracted eight emotion directions (joy, love, excitement, gratitude, sadness, anger, fear, and disgust) by mean-pooling activations from the GoEmotions dataset.
  • Denoising: They used Neutral-PCA to remove patterns common to neutral text, testing if "cleaner" emotion vectors performed better.
  • Layer Selection: An automated process identified the internal layers where emotion discrimination was strongest (Layer 14 for the 0.5B model and Layer 18 for the 3B model), as shown in Figure 2.
  • Interventions: They used Steering (adding emotion vectors to activations) and Subspace Ablation (removing the emotion-related component of an activation) to test causality.
  • Energy-Matched Controls: To ensure results weren't just due to the size of the perturbation, they compared their interventions against random directions that changed the activation by the same "energy" (norm).
Figure 2: Layer selection using emotion validation. Layers 14 and 18 are selected for Qwen2.5-0.5B and Qwen2.5- 3B, respectively.

Datasets

  • GoEmotions: Used to extract the initial emotion vectors.
  • EmpatheticDialogues: Used for cross-dataset validation during layer selection.
  • SST-2 & TweetEval: The primary benchmarks for sentiment probing and causal interventions.
  • Amazon Polarity: Used to test the "frozen transfer" of sentiment correction heads on 20,000 reviews.

Key Insights and Ideas

  • Massive Overlap: Emotion vectors and sentiment directions share a huge amount of geometric space (40% to 66% coverage). This explains why messing with "emotions" has such a strong effect on "sentiment."
  • Denoising is a Double-Edged Sword: While Neutral-PCA improved how well emotion vectors aligned with positive/negative valence, it didn't consistently improve the accuracy of sentiment probes.
  • Causal, but not Unique: Ablating the emotion subspace caused significantly more accuracy loss than random controls on TweetEval. However, the authors caution that this doesn't prove a "uniquely emotional mechanism"—it might just be that these vectors are highly efficient carriers of general sentiment information.
  • Utility vs. Mechanism: Even though emotion features can be used to "correct" and improve a model's accuracy, they often fail to outperform a simple "margin-only" baseline (which just looks at the model's log-probability gap between positive and negative tokens).

Key Results

  • Probing Accuracy: Probes using only emotion features achieved 88.66% (0.5B) and 90.70% (3B) accuracy on TweetEval, proving the information is highly accessible.
  • Excess Loss: In ablation studies, removing the raw emotion space caused 2.39 (0.5B) and 1.10 (3B) percentage points more accuracy loss than energy-matched random subspaces.
  • Steering: Denoised valence steering consistently exceeded the maximum response of 50 random directions, showing a strong causal link.
  • Correction: Adding emotion features to the model's output margin improved accuracy on Amazon Polarity by up to 1.77 points, though results were inconsistent across different model sizes and source datasets.

Limitations and Future Work

The authors acknowledge several constraints:

  • Model Scope: The study only looked at the Qwen2.5 family; results might differ for other architectures.
  • Layer Specificity: Analysis was restricted to a single "best" layer per model, potentially ignoring emotional information distributed elsewhere.
  • Dataset Sensitivity: Causal effects were much stronger on TweetEval than on SST-2, suggesting that the "emotionality" of the task matters.
  • Future Directions: The researchers suggest exploring more diverse model families and developing more sophisticated "data-aligned" controls to further isolate emotional mechanisms from general semantic ones.

Summary updated

Abstract

Emotion vectors are directions in a language model's internal representations that are associated with emotional concepts. Prior work shows that these vectors can influence model behavior, but their role in sentiment classification remains unclear. This study investigates emotion directions in Qwen2.5-0.5B-Instruct and Qwen2.5-3B-Instruct. Eight directions extracted from GoEmotions are evaluated through sentiment probing, activation interventions, and supervised correction. Emotion-only probes achieve 88.66% and 90.70% accuracy on binary TweetEval. In exploratory matched-ablation experiments, raw emotion-space removal causes 2.39 and 1.10 percentage points more accuracy loss than matched isotropic random-subspace removal for 0.5B and 3B, respectively. All four TweetEval raw and denoised effects are significant after Holm adjustment across eight comparisons. An exploratory SST-2 extension shows smaller raw excess losses of 0.66 and 0.57 points, with no effects surviving Holm adjustment. Frozen sentiment-correction classifiers are also tested on 20,000 Amazon Polarity reviews. Emotion-based correction improves accuracy over uncorrected Qwen, but does not consistently outperform margin-only baselines. These results distinguish sentiment information available in model representations, the effects of changing those representations, and their practical value for classification. They do not establish a uniquely emotional mechanism.
BibTeX
@misc{khan2026do,
  author = {Ali Hassan Khan and Saad Bin Ather and Muhammad Saif and Maryam Bashir and Hajra Waheed},
  title = {{Do Language Models Feel the Sentiment? Investigating Emotion Vectors}},
  year = {2026},
  eprint = {2611.13744},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.13744},
}
Figure