Abstract
While the general task of textual sentiment classification has been widely studied, much less research looks specifically at sentiment between a specified source and target. To tackle this problem, we experimented with a state-of-the-art relation extraction model. Surprisingly, we found that despite reasonable performance, the model's attention was often systematically misaligned with the words that contribute to sentiment. Thus, we directly trained the model's attention with human rationales and improved our model performance by a robust 4~8 points on all tasks we defined on our data sets. We also present a rigorous analysis of the model's attention, both trained and untrained, using novel and intuitive metrics. Our results show that untrained attention does not provide faithful explanations; however, trained attention with concisely annotated human rationales not only increases performance, but also brings faithful explanations. Encouragingly, a small amount of annotated human rationales suffice to correct the attention in our task.
Introduction: Beyond General Sentiment
Most sentiment analysis focuses on the author's overall opinion. However, in formal news or complex social media, we often need to extract sentiment relations: identifying who feels what about whom. For example, in the sentence "The sale infuriated Beijing," the source is "The sale," the target is "Beijing," and the sentiment is negative. Identifying these fine-grained relations requires models to understand the specific links between entities and the words that convey emotion.
The Challenge: Unfaithful Attention
State-of-the-art models often utilize an "attention" mechanism to highlight important words. However, researchers discovered a major flaw: while models might correctly identify sentiment, their internal attention is often systematically misaligned. Instead of looking at sentiment-bearing words like "infuriated," a model might focus solely on the entity name "Beijing." This makes the model's "explanation" unfaithful—it claims to be looking at one thing while actually relying on another (often entity bias) to make its prediction.
Methodology: Rationale-Augmented Training
To address this, we utilized a dual-objective training approach on an AttnLSTM architecture:
- Classification Loss: The standard goal of predicting the correct sentiment label.
- Attention Loss: A new component using KL-divergence to penalize the model when its attention distribution differs from human-provided "rationales" (the specific words a human uses to justify the sentiment).
To measure success, we introduced two novel metrics for Faithfulness: Probes-needed (how many high-attention words must be checked to find the one that actually changed the prediction) and Mass-needed (the probability weight placed on irrelevant words before hitting the important one).
Results and Discussion
The "Trained-attn" model outperformed all baselines, showing a robust improvement of 4.9 to 8.3 points in F-score across different tasks. More importantly, the model achieved drastic gains in faithfulness. On the GFBF (GoodFor/BadFor) dataset, the "probes-needed" dropped from over 8.0 (near random) to 1.55, indicating that the model's internal highlights finally matched the words it was actually using to calculate sentiment.
The "Nudge" Effect
One of the most encouraging findings was that a massive dataset of human rationales is not required. The benefits are submodular; the first few human annotations provide the most significant boost. Just 100 rationales (roughly 4-7% of the total) provided nearly half of the total performance gain.
Conclusion
Attention is not inherently an explanation. While untrained attention is often unfaithful, "nudging" it with concise human rationales can align a model's internal logic with human reasoning. This not only improves predictive accuracy but also ensures that the model is getting the right answer for the right reasons. Future work will focus on maintaining this faithfulness in more complex, compositional language where sentiment is not tied to a single keyword.
Summary updated