More than Words: Syntactic Packaging and Implicit Sentiment

Stephan Greene, Philip ResnikNAACL 2009

TL;DR

This paper introduces a novel approach to sentiment analysis that moves beyond explicit opinion words to detect "implicit sentiment" conveyed through sentence structure. By identifying "observable proxies for underlying semantics" (OPUS) derived from syntactic dependencies, they achieve significant improvements in sentiment classification accuracy, outperforming existing state-of-the-art methods on challenging datasets.

What Problem Does This Paper Address?

Traditional sentiment analysis often focuses on identifying explicit opinion-bearing words and phrases. However, sentiment can also be conveyed subtly through the way information is "packaged" in a sentence, even when the language appears objective. For instance, the difference between "a soldier veered his jeep into a market and killed three civilians" versus "a soldier's jeep veered into a market, causing three civilian deaths" evokes different feelings towards the soldier, despite using similar words. This paper tackles the challenge of detecting this "implicit sentiment" – the subtle emotional coloring or perspective embedded in sentence structure, which is crucial for a deeper understanding of text.

Methods

The authors propose a method that leverages linguistic theory on "grammatically relevant semantic properties" (like volition, causation, and change of state) that influence how events are described syntactically. They hypothesize that syntactic choices manipulate these semantic properties, thereby conveying implicit sentiment.

Their core innovation is the development of Observable Proxies for Underlying Semantics (OPUS). Instead of directly annotating abstract semantic properties (which is difficult and time-consuming), OPUS features are derived from observable grammatical relations (syntactic dependencies) between words identified as relevant to a specific domain. These features capture how terms are used in relation to each other within a sentence, acting as proxies for the underlying semantic properties.

For classification, they employ a standard supervised learning approach using Support Vector Machines (SVMs). They compare their OPUS features against baseline methods that rely on simpler features like word n-grams.

Datasets

  • Death Penalty (DP) Corpus: A newly constructed corpus for experimentation, comprising documents from pro- and anti-death-penalty websites, balanced to have an equal number of documents per side (596 documents per side).
  • Bitter Lemons Corpus: A publicly available corpus of essays presenting Israeli and Palestinian viewpoints, used for direct comparison with prior state-of-the-art work. It contains 297 documents from each perspective.

Key Insights and Ideas

  • Structure Carries Sentiment: The paper provides strong empirical evidence that sentence structure, not just word choice, significantly influences perceived sentiment.
  • Grammatical Properties as Mediators: Grammatically relevant semantic properties (like volition, causation, change of state) act as a bridge between syntactic choices and implicit sentiment.
  • OPUS: A Practical Bridge: OPUS features offer a computationally tractable way to approximate these abstract semantic properties from observable syntactic information, making implicit sentiment analysis feasible.
  • Domain-Specific Relevance: The effectiveness of OPUS features is tied to identifying domain-relevant verbs, suggesting that the specific vocabulary of a domain plays a role in how sentiment is expressed syntactically.

Key Results

  • Significant Accuracy Gains: Using OPUS features, the authors achieved substantial and statistically significant improvements in sentiment classification accuracy on the DP corpus. For example, with 7552 features, their accuracy reached 88.10%, compared to a baseline of 71.96%.
  • Outperforming State-of-the-Art: On the Bitter Lemons corpus, their OPUS-based SVM classifier achieved an average accuracy of 95.41%, with a best accuracy of 97.64%. This outperformed existing state-of-the-art SVM and Naive Bayes classifiers (Lin et al., 2006).
  • Validation of Linguistic Theory: The empirical validation study confirmed that syntactic forms (transitive vs. nominalized) significantly impact perceived sympathy towards a perpetrator, aligning with linguistic theories of framing.
  • Importance of "Kill" Verbs: Experiments showed that OPUS features derived from "kill" verbs were particularly effective, highlighting the impact of specific verb classes on sentiment.

Limitations and Future Work

  • Feature Engineering: The OPUS approach relies on identifying domain-relevant terms and a threshold for their relative frequency, which are experimental parameters.
  • Generalizability: While the approach shows promise across different domains (death penalty, political conflict), further investigation is needed to assess its broad applicability.
  • Future Work: The authors suggest that these features could also improve conventional opinion labeling for subjective text, which they plan to investigate. They also aim to explore how these features might be used in other computational linguistics tasks.

Summary updated

BibTeX
@inproceedings{greene2009more,
  author = {Stephan Greene and Philip Resnik},
  title = {{More than Words: Syntactic Packaging and Implicit Sentiment}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2009},
}
Figure