Review Sentiment Scoring via a Parse-and-Paraphrase Paradigm

Jingjing Liu, Stephanie SeneffEMNLP 2009 · OpenReview

Abstract

This paper presents a parse-and-paraphrase paradigm to assess the degrees of sentiment for product reviews. Sentiment identification has been well studied; however, most previous work provides binary polarities only (positive and negative), and the polarity of sentiment is simply reversed when a negation is detected. The extraction of lexical features such as unigram/bigram also complicates the sentiment classification task, as linguistic structure such as implicit long-distance dependency is often disregarded. In this paper, we propose an approach to extracting adverb-adjective-noun phrases based on clause structure obtained by parsing sentences into a hierarchical representation. We also propose a robust general solution for modeling the contribution of adverbials and negation to the score for degree of sentiment. In an application involving extracting aspect-based pros and cons from restaurant reviews, we obtained a 45% relative improvement in recall through the use of parsing methods, while also improving precision.

Introduction

Traditional sentiment analysis often relies on "shallow" features like word counts or bigrams. While effective for simple classification, these methods suffer from two major flaws: structural blindness and the "negation flip" fallacy. Shallow models struggle with long-distance dependencies—for instance, missing a negation at the start of a sentence that modifies a word at the end. Furthermore, many systems handle negation by simply reversing polarity (e.g., flipping a +5 score to -5). This fails to capture linguistic nuance, where "not bad" is distinct from "excellent," and "not very clean" is not the same as "filthy."

This research introduces a parse-and-paraphrase paradigm designed to extract specific "aspects" of a review (e.g., food, service, price) and assign them nuanced numerical scores rather than simple binary labels.

Methodology

The proposed system utilizes a three-stage pipeline to process unstructured review text into structured sentiment summaries.

The three-stage system architecture: Data Filtering, Topic Extraction, and Sentiment Scoring.
  1. Parse-and-Paraphrase: The system employs a lexicalized probabilistic syntactic grammar to generate a "linguistic frame." This frame captures the hierarchical relationship between adverbs, adjectives, and nouns, allowing the system to identify exactly which clause a negation belongs to. These are then paraphrased into clean triplets (e.g., really-good-caesar).
  2. LM-based Topic Clustering: To group specific words into broader categories (e.g., mapping "pizza" to "Food"), the authors use a Language Model (LM) approach based on bigram probabilities. The system also handles "widow-adjectives"—adjectives appearing without a noun—by statistically associating them with surrogate topics like "Price."
  3. Linear Additive Model: The core scoring logic treats adjectives as base scores derived from average user ratings. Adverbs and negations are then applied as "universal scales" or linear offsets. For example, a negation like "not" is modeled as a significant negative subtraction rather than a simple sign reversal.

Experimental Results

The system was evaluated using a massive crawl of 137,569 restaurant reviews from Citysearch. The "Pros/Cons" sections of these reviews served as the gold standard for aspect extraction.

Performance comparison showing recall and precision for different models with and without clustering.

As shown in the performance data, the linguistic parsing approach (LING) achieved a 45% relative improvement in recall over the neighbor baseline (NB) when extracting aspects. When compared to human annotators on a 1–5 sentiment scale, the system’s scores were remarkably accurate, with a mean distance of only 0.43 to 0.46. The system also achieved a high level of polarity agreement with humans, reaching a Kappa score of 0.55–0.60.

Conclusion

By treating negation as a linear offset and utilizing hierarchical parsing to maintain linguistic scope, this paradigm significantly outperforms shallow feature models. While the current implementation is tuned to the restaurant domain and faces some challenges with unparsable "noisy" text (approximately 21.4% of the dataset), the underlying logic of universal adverbial scales provides a robust foundation for future multimodal and multi-domain sentiment applications.

Summary updated

Abstract

This paper presents a parse-and-paraphrase paradigm to assess the degrees of sentiment for product reviews. Sentiment identification has been well studied; however, most previous work provides binary polarities only (positive and negative), and the polarity of sentiment is simply reversed when a negation is detected. The extraction of lexical features such as unigram/bigram also complicates the sentiment classification task, as linguistic structure such as implicit long-distance dependency is often disregarded. In this paper, we propose an approach to extracting adverb-adjective-noun phrases based on clause structure obtained by parsing sentences into a hierarchical representation. We also propose a robust general solution for modeling the contribution of adverbials and negation to the score for degree of sentiment. In an application involving extracting aspect-based pros and cons from restaurant reviews, we obtained a 45% relative improvement in recall through the use of parsing methods, while also improving precision.
BibTeX
@inproceedings{liu2009review,
  author = {Jingjing Liu and Stephanie Seneff},
  title = {{Review Sentiment Scoring via a Parse-and-Paraphrase Paradigm}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2009},
  url = {https://openreview.net/forum?id=rJbWIMM_br},
}
Figure