BuildingWord-Emotion Mapping Dictionary for Online News

Yanghui Rao, Xiaojun Quan, Liu Wenyin, Qing Li, Mingliang Chen

TL;DR

Researchers have developed an automated, language-independent method to build "word-emotion mapping dictionaries" that go beyond simple positive/negative sentiment. By applying maximum likelihood estimation to news articles and their associated reader ratings, the system learns to associate specific words with a distribution of fine-grained emotions like "amusement," "touched," or "anger."

The Challenge: Beyond Positive and Negative

Most sentiment analysis tools are "coarse-grained"—they tell you if a text is positive, negative, or neutral. While this works for product reviews, it fails to capture the nuance of online news. A news article might be written in a perfectly objective, neutral tone, yet evoke a powerful emotional response in the reader (e.g., a story about a tragedy or a heartwarming rescue).

Existing emotional lexicons (like WordNet-Affect or SentiWordNet) have three major flaws:

  1. Domain Mismatch: A word’s emotional weight in a general dictionary might not fit a specific news context.
  2. Language Barriers: Most high-quality lexicons are English-only.
  3. Static Nature: They can’t easily update to include new slang or changing social contexts.

The authors sought a way to build these dictionaries automatically, without human annotators, using the "social emotions" (user ratings) already present on many news portals.

Methodology: Learning from the Crowd

The core of the paper is a mathematical framework for building a $W \times E$ matrix, where $W$ is the vocabulary and $E$ is a set of predefined emotions.

  • Maximum Likelihood Estimation (MLE): The authors model the probability of a reader feeling a certain emotion given a specific word. They use Jensen’s inequality to simplify the complex log-likelihood of the data, allowing them to calculate the word-emotion mappings efficiently.
  • The "Background Word" Problem: Not every word in a news article is emotional. Words like "report," "yesterday," or "official" appear frequently but carry little emotional weight. The authors define these as background words.
  • Pruning Strategies: To refine the dictionary, they introduced three parameter-free pruning algorithms ($rtf-max$, $rtf-ave$, and $rtf-min$). These compare the probability of a word being "background noise" versus its probability of carrying emotional weight.
Table showing the relationship between training document volume, vocabulary size, and pruning thresholds.

Datasets and Evaluation

The researchers tested their approach on two very different datasets to prove it was language-independent:

  1. Sina (Chinese): A massive dataset of 32,493 news articles from Sina.com, including user ratings across 8 emotions: touched, empathy, boredom, anger, amusement, sadness, surprise, and warmness.
  2. SemEval-2007 (English): A smaller, standard benchmark consisting of 1,246 news headlines with scores for 6 emotions: anger, disgust, fear, joy, sadness, and surprise.
Example of the social emotion rating interface used by readers to tag news articles.

Key Insights

  • Readers as Annotators: Instead of paying experts to label words, the model treats millions of casual news readers as a "distributed" labeling workforce. Their aggregate clicks on emotion buttons provide the ground truth.
  • Pruning is Essential: One of the most surprising findings was that deleting more than 30% of the vocabulary (the "background words") actually improved the model's accuracy. By removing the noise, the emotional signals of the remaining words became much clearer.
  • Context Matters: The authors highlight that news is "objective" but "implicitly emotional." Their method successfully bridges the gap between factual reporting and the human response.

Key Results and Limitations

On the Sina dataset, the model showed a strong Pearson correlation between predicted and actual emotional distributions. The $rtf-min$ pruning strategy was the clear winner, significantly outperforming unpruned dictionaries. The model was particularly good at identifying the "dominant" emotion (the one with the most user votes).

However, the paper notes a significant drop in performance when dealing with short text. On the SemEval dataset (headlines only), the average correlation was much lower (0.36). This is due to data sparsity: a five-word headline simply doesn't provide enough context for the dictionary to work effectively. Future research should focus on improving emotional annotation for short-form text.

Summary updated

Abstract

Sentiment analysis of online documents such as news articles, blogs and microblogs has received increasing attention. We propose an efficient method of automatically building the word-emotion mapping dictionary for social emotion detection. In the dictionary, eachwordis associated with the distribution on a series of human emotions. In addition, three different pruning strategies are proposed to refine the dictionary. Experiment on the real-world data sets has validated the effectiveness and reliability of the method. Compared with other lexicons, the dictionary generated using our approach is more adaptive for personalized data set, language-independent, fine-grained, and volume-unlimited. The generated dictionary has a wide range of applications, including predicting the emotional distribution of news articles and tracking the change of social emotions on certain events over time.
BibTeX
@inproceedings{rao2012buildingword,
  author = {Yanghui Rao and Xiaojun Quan and Liu Wenyin and Qing Li and Mingliang Chen},
  title = {{BuildingWord-Emotion Mapping Dictionary for Online News}},
  booktitle = {SDAD@ECML/PKDD},
  year = {2012},
  url = {https://openreview.net/forum?id=-80VNngWrc9},
}
Figure