Good News or Bad News: Using Affect Control Theory to Analyze Readers’ Reaction Towards News Articles

Areej Alhothali, Jesse HoeyNAACL 2015

TL;DR

This paper introduces a novel approach to sentiment analysis by applying Affect Control Theory (ACT) to understand reader reactions to objective news articles. By modeling emotions in a three-dimensional space and leveraging a newly expanded lexicon, the method achieves significantly higher accuracy than traditional sentiment analysis techniques in predicting sentiment towards news events, subjects, and objects.

What Problem Does This Paper Address?

Traditional sentiment analysis often focuses on subjective texts, aiming to classify them as positive or negative. However, many real-world texts, such as news articles, are objective and factual, yet they still evoke emotions and sentiments in readers. Analyzing these objective statements and their components (like the actors and actions described) presents a challenge. Existing methods struggle to capture the nuanced emotional responses that objective news can elicit, especially when considering the interplay between fundamental cultural sentiments and the transient impressions formed by specific events. This paper aims to bridge this gap by developing a method to predict reader sentiment towards both the overall event and its constituent parts within objective news articles.

Methods

The core of this paper's approach is the application of Affect Control Theory (ACT), a social psychological theory that models how people form impressions and emotions based on their fundamental sentiments about social situations and the transient impressions from events.

Here's a breakdown of their key methods:

  • Affect Control Theory (ACT): ACT posits that emotions arise from the difference between fundamental cultural sentiments and transient impressions formed by events. These sentiments are represented in a three-dimensional space: Evaluation (good/bad), Potency (powerful/powerless), and Activity (lively/inactive), often referred to as EPA.
  • Event Modeling: Events are modeled as subject-verb-object (SVO) triplets. The theory uses empirically derived equations to predict the transient impression (T) of an event based on the fundamental sentiments (f) of its components.
  • Lexicon Expansion: The existing ACT lexicon, containing a few thousand words, was significantly expanded. This was achieved by:
  • Combining existing lexicons (original ACT, ANEW, Warriner et al.).
  • Employing a graph-based semi-supervised learning method called label propagation. This technique leverages WordNet similarities to assign EPA values to new words, effectively expanding the lexicon to over 17,000 words.
  • Sentiment Prediction: For each news headline, the system extracts the subject, verb, and object (and sometimes modifiers and settings). It then uses the augmented ACT lexicon and ACT's impression formation equations to compute the predicted emotion towards the event, subject, and object.
  • Emotion Mapping: The predicted EPA scores for emotions are mapped to the nearest emotion label in ACT using a Euclidean distance measure.

Datasets

  • Augmented ACT Lexicon: The authors created an expanded lexicon by combining existing ACT lexicons with ANEW and Warriner et al. data, and then further augmented it using label propagation. This resulted in a lexicon of 17,347 words.
  • News Headlines Dataset: A new dataset of 2080 news headlines was collected from various news websites and archives (BBC, CNN, Reuters, The Telegraph, Times) spanning from 1999 to 2014. This dataset was manually annotated by at least three judges per headline, who provided EPA ratings for the subject, behavior, object, and the overall event. After screening, 1658 headlines were used for evaluation.

Key Insights and Ideas

  • Objective Texts Evoke Emotions: The paper strongly demonstrates that even objective, factual statements in news headlines can evoke distinct and predictable emotional responses in readers. This challenges the traditional focus of sentiment analysis on explicitly subjective content.
  • ACT as a Powerful Framework for Nuanced Sentiment: Affect Control Theory, with its multi-dimensional EPA space and focus on fundamental cultural sentiments, provides a richer and more nuanced way to represent and predict emotions compared to simple positive/negative polarity.
  • Lexicon Expansion is Crucial: The significant expansion of the ACT lexicon using label propagation was key to achieving higher accuracy, showing the importance of comprehensive affective lexicons for sentiment analysis.
  • Structure Matters: Subject-Verb-Object Analysis: By decomposing sentences into SVO triplets and analyzing the roles of each component, the model can predict distinct sentiments towards the actor, the action, and the recipient, offering a more granular understanding than document-level sentiment.
  • Deflection as an Indicator of Expectation: The concept of "deflection" in ACT (the discrepancy between fundamental and transient sentiments) is shown to be a useful indicator of how unexpected an event is, providing further insight into reader reactions.

Key Results

  • Headline Sentiment Classification: The proposed ACT-based method (ACT-PTQ, which uses parse tree quintets) achieved a precision of 82% in classifying the overall sentiment of news headlines, significantly outperforming a standard sentiment classifier (STD-classifier) which achieved 57% precision.
  • Subject and Object Sentiment Prediction: The model achieved precisions of 79% for predicting sentiment towards the subject (actor) and 68% towards the object.
  • RMSE and MAS: For predicting emotions towards the subject and object, RMSE and MAS values were generally less than 1.5, indicating a good agreement between predicted and ground truth EPA scores.
  • Qualitative Examples: The paper provides compelling examples (e.g., "Russia says 4 militants killed in Dagestan siege" evoking "furious" towards the subject and "sorry" towards the object) that illustrate the nuanced emotional predictions.

Limitations and Future Work

  • Limitations:
    • The label propagation algorithm's performance for adverbs was limited due to fewer adverbs in the ACT lexicon and the WordNet similarity measure's focus on words of the same part-of-speech.
  • The current model primarily extracts single subject, verb, and object. It does not fully account for adverbs, phrasal verbs, negations, or the accumulation of emotions from multiple consequent behaviors within a sentence.
  • The grammar of news headlines can be ungrammatical and imprecise, posing challenges for standard parsing techniques.
  • Future Work:
    • Improve the model by incorporating adverbs, phrasal verbs, negations, and handling multiple behaviors within a sentence.
  • Explore more complex parse trees and accumulate emotions from sequential behaviors.
  • Investigate using ACT predictions to bootstrap supervised learning methods.
  • Augment the method with more complex levels of detail and gather more extensive datasets for evaluation.
  • Evaluate ACT for more precise and detailed sentiment analysis.

Summary updated

BibTeX
@inproceedings{alhothali2015good,
  author = {Areej Alhothali and Jesse Hoey},
  title = {{Good News or Bad News: Using Affect Control Theory to Analyze Readers’ Reaction Towards News Articles}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2015},
}
Figure