The Importance of Pronouns to Sentiment Analysis: Online Cancer Survivor Network Case Study

Nir Ofek, Lior Rokach, Cornelia Caragea, John YenWWW 2015 · DOI

TL;DR

This paper investigates the impact of pronoun usage on sentiment analysis within an online cancer survivor community. They found that including pronouns significantly improves sentiment classification accuracy, suggesting that these seemingly simple words carry valuable insights into user emotions and personality.

What Problem Does This Paper Address?

Sentiment analysis, the process of determining the emotional tone of text, is crucial for understanding user needs and concerns in online communities. While standard text classification often removes "function words" like pronouns to simplify models, this paper questions whether this is always the best approach, especially for tasks that rely on understanding subjectivity and personality. The authors hypothesize that pronouns, due to their inherent connection to the writer's perspective, might be particularly informative for sentiment analysis, especially in contexts where emotional states and personal traits are important. They aim to quantify this impact using a real-world case study.

Methods

The researchers employed a lexical approach to sentiment analysis, focusing on the role of pronouns. They analyzed posts from the Cancer Survivors Network (CSN) and calculated the frequency ratios of pronouns in positive versus negative sentiment posts. To evaluate the impact of pronouns, they trained sentiment classifiers using two feature sets: one including all words and another excluding pronouns. They then compared the performance of these classifiers using 10-fold cross-validation. Specifically, they used three features derived from pronoun frequency ratios: the maximum ratio, the sum of the top three ratios, and the sum of the top six ratios. They also compared the pronoun usage in the CSN dataset to general English usage.

Datasets

The primary dataset used in this study is the Cancer Survivors Network (CSN), a large online community for cancer patients and their caregivers. The authors utilized a subset of forum posts collected from July 2000 to October 2010. For training a sentiment classifier, they manually annotated 298 posts from the CSN breast cancer forum, classifying them as either positive (204 posts) or negative (94 posts).

Key Insights and Ideas

  • Pronouns are not just filler words: Contrary to the common practice of removing function words in text classification, this study demonstrates that pronouns are significant indicators of sentiment.
  • Personal perspective matters: The unique usage patterns of pronouns in self-descriptions and emotional expressions can reveal a writer's subjective attitude and emotional state.
  • Community-specific language: Pronoun usage in the CSN dataset differs from standard English, highlighting the need for domain-specific analysis in online communities.
  • Quantifiable impact: The inclusion of pronouns leads to measurable improvements in sentiment classification performance across various algorithms.

Key Results

The study found a consistent drop in sentiment classification performance when pronouns were omitted from the text. For instance, the adaboost LMT classifier showed a 5.84% drop in AUC and a 0.14% drop in F-measure when pronouns were excluded. While the drop in Accuracy was less pronounced, the overall trend indicated that pronouns contribute positively to sentiment classification. A two-way ANOVA confirmed a significant effect of pronouns on sentiment classification for AUC and Accuracy. The analysis also revealed that pronoun frequencies in the CSN dataset differ from those in general English, with pronouns like "us," "you," and "they" showing higher positive-to-negative frequency ratios in the CSN.

Limitations and Future Work

The authors acknowledge that while their findings provide insights into the importance of pronouns for sentiment analysis, they also raise questions about the deeper psychological meanings and dimensions of these findings. Future work could explore these psychological underpinnings further. Additionally, the study focused on a specific online health community; extending this analysis to other domains and types of online communities could provide broader insights. The authors also suggest investigating how pronoun usage might evolve over time within a community or in response to specific events.

Summary updated

BibTeX
@inproceedings{ofek2015importance,
  author = {Nir Ofek and Lior Rokach and Cornelia Caragea and John Yen},
  title = {{The Importance of Pronouns to Sentiment Analysis: Online Cancer Survivor Network Case Study}},
  booktitle = {The Web Conference},
  year = {2015},
  doi = {10.1145/2740908.2742781},
}

Loading related content...

Figure