Learning with Compositional Semantics as Structural Inference for Subsentential Sentiment Analysis

Yejin Choi, Claire CardieEMNLP 2008 · OpenReview

Abstract

Determining the polarity of a sentiment-bearing expression requires more than a simple bag-of-words approach. Words or constituents within an expression interact to yield a specific overall polarity. This paper presents a novel learning-based approach that incorporates structural inference motivated by compositional semantics into the learning procedure. Experiments demonstrate that while simple compositional heuristics can outperform standard learning methods (89.7% vs. 89.1% accuracy), integrating compositional semantics directly into the learning process achieves the best performance (90.7%). Additionally, the study highlights the importance of "content-word negators" and finds that extra context outside expression boundaries can actually decrease classification accuracy.

Introduction

Traditional sentiment analysis often relies on "bag-of-words" models, where the overall sentiment is treated as a sum of individual word polarities. However, language is inherently structural. In a phrase like "I have no doubt," the negative word "doubt" is flipped by the negator "no" to create a positive sentiment. Capturing these interactions is critical for accurate subsentential sentiment analysis.

A significant challenge lies in the variety of negators. Beyond simple function words like "not," "content-word negators" such as "eliminate" or "lack" also flip polarity (e.g., "eliminated the risk"). Standard learning models using flat feature vectors often fail to represent the recursive, structural nature of these linguistic compositions. This research bridges the gap by treating sentiment as a structural inference problem grounded in compositional semantics.

Methodology: Classification with Compositional Inference (CCI)

The authors propose Classification with Compositional Inference (CCI), a framework that introduces hidden variables for each token to represent its local semantic role—whether it is a positive word, a negative word, or a negator.

Compositional Rules

The core of CCI is a set of compositional rules that define how different parts of speech interact. For example, a Verb Phrase (VP) followed by a Noun Phrase (NP) might have its polarity determined by the composition of the two.

Compositional rules and examples defining how constituents interact to determine expression polarity.

Learning and Inference

To train the model without manual labels for every word's hidden state, the authors utilize a "soft gold standard" initialized from existing lexicons. This process assigns initial roles to tokens based on their presence in negator or polarity lexicons.

Algorithm for constructing the soft gold standard for hidden variables based on lexicon lookups.

The model is trained using a modified Margin Infused Relaxed Algorithm (MIRA). During training, the system searches for the configuration of hidden variables that best predicts the overall sentiment label. This allows the model to learn to "correct" initial lexicon-based assignments if they lead to incorrect global predictions, effectively learning the nuances of composition from data.

Experimental Results

The proposed methods were evaluated using the MPQA (Multi-Perspective Question Answering) corpus, which contains 535 newswire documents annotated with phrase-level sentiment. The focus was on "strong" sentiment expressions with clearly defined boundaries.

Performance Comparison

The CCI approach achieved the highest accuracy, outperforming both simple baselines and purely rule-based systems:

  • Simple Voting (Baseline): 86.5%
  • Simple Learning (SC-NEGEX): 89.1%
  • Compositional Heuristics (COMPO): 89.7%
  • Compositional Learning (CCI-COMPO): 90.7%

The improvement of CCI-COMPO over standard learning methods is statistically significant, demonstrating that structural inference provides a measurable advantage over flat feature-based learning.

Key Insights

  • The Role of Content-Word Negators: Words like "eliminate," "lack," and "minimal" act as negators. Incorporating these "content-word negators" significantly boosts performance compared to models that only consider function words like "not."
  • Structure vs. Raw Data: Interestingly, simple heuristic rules based on compositional semantics (89.7%) outperformed standard machine learning models that lacked structural awareness (89.1%).
  • Context Sensitivity: Contrary to conventional wisdom, expanding the context beyond the identified expression boundaries decreased classification accuracy. This suggests that sentiment is highly localized within the expression, and extra context often introduces noise.

Conclusion

Integrating compositional semantics into a learning framework provides a robust solution for subsentential sentiment analysis. While the current rules are manually defined, future work could explore learning these compositional rules directly from data. Furthermore, improving the automatic extraction of opinion expression boundaries remains a key challenge for applying these methods to unannotated text.

Summary updated

Abstract

Determining the polarity of a sentiment-bearing expression requires more than a simple bag-of-words approach. In particular, words or constituents within the expression can interact with each other to yield a particular overall polarity. In this paper, we view such subsentential interactions in light of compositional semantics, and present a novel learning-based approach that incorporates structural inference motivated by compositional semantics into the learning procedure. Our experiments show that (1) simple heuristics based on compositional semantics can perform better than learning-based methods that do not incorporate compositional semantics (accuracy of 89.7% vs. 89.1%), but (2) a method that integrates compositional semantics into learning performs better than all other alternatives (90.7%). We also find that "content-word negators", not widely employed in previous work, play an important role in determining expression-level polarity. Finally, in contrast to conventional wisdom, we find that expression-level classification accuracy uniformly decreases as additional, potentially disambiguating, context is considered.
BibTeX
@inproceedings{choi2008learning,
  author = {Yejin Choi and Claire Cardie},
  title = {{Learning with Compositional Semantics as Structural Inference for Subsentential Sentiment Analysis}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2008},
  url = {https://openreview.net/forum?id=rkWzUfzuZS},
}
Figure