TL;DR
This paper investigates whether "emotion vectors"—specific directions in a language model's internal representation—are the primary drivers behind sentiment classification. By analyzing Qwen2.5 models, the researchers found that while these vectors contain significant sentiment information and causally influence model behavior, they do not necessarily represent a unique or superior mechanism compared to simpler confidence-based baselines.
What Problem Does This Paper Address?
Large Language Models (LLMs) are remarkably good at sentiment analysis, but the internal mechanics of how they arrive at these labels remain opaque. We know LLMs encode emotional concepts (like "joy" or "anger"), but does the model actually use these emotional representations to decide if a tweet is positive or negative?
The authors aim to disentangle three distinct concepts:
- Accessible Information: Can we find sentiment info hidden in emotion-related activations?
- Causal Influence: Does changing these activations actually change the model's output?
- Predictive Utility: Does knowing these emotional features help us build better sentiment classifiers than just looking at the model's final output?
Methods
The study focuses on two models: Qwen2.5-0.5B-Instruct and Qwen2.5-3B-Instruct. The workflow is summarized in Figure 1.
Key methodological steps included:
- Vector Extraction: The authors extracted eight emotion directions (joy, love, excitement, gratitude, sadness, anger, fear, and disgust) by mean-pooling activations from the GoEmotions dataset.
- Denoising: They used Neutral-PCA to remove patterns common to neutral text, testing if "cleaner" emotion vectors performed better.
- Layer Selection: An automated process identified the internal layers where emotion discrimination was strongest (Layer 14 for the 0.5B model and Layer 18 for the 3B model), as shown in Figure 2.
- Interventions: They used Steering (adding emotion vectors to activations) and Subspace Ablation (removing the emotion-related component of an activation) to test causality.
- Energy-Matched Controls: To ensure results weren't just due to the size of the perturbation, they compared their interventions against random directions that changed the activation by the same "energy" (norm).
Datasets
- GoEmotions: Used to extract the initial emotion vectors.
- EmpatheticDialogues: Used for cross-dataset validation during layer selection.
- SST-2 & TweetEval: The primary benchmarks for sentiment probing and causal interventions.
- Amazon Polarity: Used to test the "frozen transfer" of sentiment correction heads on 20,000 reviews.
Key Insights and Ideas
- Massive Overlap: Emotion vectors and sentiment directions share a huge amount of geometric space (40% to 66% coverage). This explains why messing with "emotions" has such a strong effect on "sentiment."
- Denoising is a Double-Edged Sword: While Neutral-PCA improved how well emotion vectors aligned with positive/negative valence, it didn't consistently improve the accuracy of sentiment probes.
- Causal, but not Unique: Ablating the emotion subspace caused significantly more accuracy loss than random controls on TweetEval. However, the authors caution that this doesn't prove a "uniquely emotional mechanism"—it might just be that these vectors are highly efficient carriers of general sentiment information.
- Utility vs. Mechanism: Even though emotion features can be used to "correct" and improve a model's accuracy, they often fail to outperform a simple "margin-only" baseline (which just looks at the model's log-probability gap between positive and negative tokens).
Key Results
- Probing Accuracy: Probes using only emotion features achieved 88.66% (0.5B) and 90.70% (3B) accuracy on TweetEval, proving the information is highly accessible.
- Excess Loss: In ablation studies, removing the raw emotion space caused 2.39 (0.5B) and 1.10 (3B) percentage points more accuracy loss than energy-matched random subspaces.
- Steering: Denoised valence steering consistently exceeded the maximum response of 50 random directions, showing a strong causal link.
- Correction: Adding emotion features to the model's output margin improved accuracy on Amazon Polarity by up to 1.77 points, though results were inconsistent across different model sizes and source datasets.
Limitations and Future Work
The authors acknowledge several constraints:
- Model Scope: The study only looked at the Qwen2.5 family; results might differ for other architectures.
- Layer Specificity: Analysis was restricted to a single "best" layer per model, potentially ignoring emotional information distributed elsewhere.
- Dataset Sensitivity: Causal effects were much stronger on TweetEval than on SST-2, suggesting that the "emotionality" of the task matters.
- Future Directions: The researchers suggest exploring more diverse model families and developing more sophisticated "data-aligned" controls to further isolate emotional mechanisms from general semantic ones.
Summary updated