Last but Definitely Not Least: On the Role of the Last Sentence in Automatic Polarity-Classification

Israela Becker, Vered AharonsonACL 2010

Introduction

In the field of Natural Language Processing (NLP), efficiency is often synonymous with algorithmic optimization. However, the paper "Last but Definitely Not Least: On the Role of the Last Sentence in Automatic Polarity-Classification" (ACL 2010) suggests that efficiency can also be found through the lens of discourse linguistics. While traditional topic extraction typically focuses on the beginning of a text, this research demonstrates that sentiment analysis—or polarity classification—should look to the end.

The Sentiment Location Problem

As digital text volumes explode, the computational cost of sentiment analysis on massive datasets becomes a significant hurdle. Traditional information retrieval models rely on the "Topic Sentence," usually found at the start of a paragraph, to determine what a text is about. The authors hypothesize that sentiment works differently: reviewers often summarize their feelings in a "bottom line" at the conclusion of their text. This "Evaluative Discourse Topic" serves as a recapitulation of the reviewer's message, potentially offering a computational shortcut for efficient sentiment detection.

Methodology

To validate this hypothesis, the researchers conducted psycholinguistic experiments using a subset of the Blitzer, Dredze, and Pereira (2007) Amazon product review database. They selected 16 reviews (8 positive, 8 negative) across four domains: Kitchen, Books, DVDs, and Electronics.

Two primary experiments were conducted:

  1. Whole-Text Rating: 35 subjects rated the polarity of full reviews on a 1–5 star scale.
  2. Single-Sentence Rating: 36 subjects rated individual sentences. The study compared the last sentence of a review against the second sentence, which served as a proxy for a standard, non-initial sentence.

The researchers measured decision latencies and mouse trajectories to gauge human certainty and effort. They then calculated Pearson correlation coefficients to determine how well these segments predicted the "ground truth" (the original author's rating).

Key Results

The quantitative findings strongly support the "last sentence" hypothesis. The correlation between the author's original rating and the reader's rating for the last sentence was remarkably high, nearly matching the performance of reading the entire review.

Pearson Correlation Coefficients comparing reader ratings of different text segments against the author's original intent.

As shown in the table above, the correlation for the last sentence (0.7616) far exceeds that of the second sentence (0.4705). This suggests that the final sentence captures the essence of the reviewer's sentiment with significantly higher fidelity than other internal sentences.

Furthermore, the distribution of rating differences (the delta between the author's rating and the reader's rating) illustrates the reliability of the last sentence as a predictor.

Distributions of the difference between author and reader ratings for (a) whole reviews, (b) last sentences, and (c) second sentences.

In the histograms above, the distribution for the last sentence (b) closely mirrors the distribution for the whole review (a), with a sharp peak at zero (indicating perfect agreement). In contrast, the second sentence (c) shows a much broader distribution, indicating higher variance and less predictive power.

Other notable findings include:

  • Decision Speed: When readers were both fast and accurate, 96% of the time they were evaluating the last sentence.
  • Latency: Average response times were shorter for last sentences (19.6s) compared to second sentences (22.1s), suggesting the sentiment signal in the final position is more "decisive" and easier to process.

Discussion and Future Work

The study highlights the "Recency Effect"—a psychological phenomenon where information at the end of a sequence is more cognitively prominent. In customer reviews, the first sentence often introduces the product (e.g., "I bought this toaster last week..."), while the last sentence provides the final recommendation (e.g., "All in all, I would recommend it highly").

While the sample size was intentionally small to ensure high-quality psycholinguistic data, the results provide a compelling case for "Efficiency through Selection." By focusing computational resources on the final sentence, developers can reduce processing costs without a substantial loss in accuracy. Future research aims to scale these experiments to hundreds of subjects and map the "polarity profile" across all sentence positions in a text.

Summary updated

BibTeX
@inproceedings{becker2010last,
  author = {Israela Becker and Vered Aharonson},
  title = {{Last but Definitely Not Least: On the Role of the Last Sentence in Automatic Polarity-Classification}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2010},
}
Figure