Feature Sentiment Diversification of User Generated Reviews: The FREuD Approach

Nasir Naveed, Thomas Gottron, Steffen Staab

Abstract

Online user reviews are essential for consumer decision-making, yet the sheer volume of content often leads to information overload. Current platforms typically rank reviews by "helpfulness" votes, which often results in redundant information and popularity bias. We propose FREuD (Feature-sentiment Diversification), an approach that combines Latent Dirichlet Allocation (LDA) for feature extraction with ANEW-based sentiment analysis. By using a greedy selection algorithm, FREuD identifies a small subset of reviews that maximizes coverage of both product features and the spectrum of user sentiments. Our experiments on real-world product reviews demonstrate that FREuD significantly outperforms standard helpfulness-based rankings and naive star-rating diversification.

Introduction: The Challenge of Review Overload

When a product accumulates hundreds or thousands of reviews, users struggle to find a representative overview. Most e-commerce sites address this by ranking reviews based on user-provided "helpfulness" scores. However, this approach suffers from two critical flaws:

  1. Popularity Bias: Early reviews accumulate more votes simply due to longer exposure, not necessarily higher quality.
  2. Redundancy: The top-ranked reviews often highlight the same popular features (e.g., battery life), leaving other critical aspects (e.g., software stability) unaddressed.

The goal of diversification is to select a subset of $k$ reviews that provides the most comprehensive "bang for your buck," covering diverse features and a balanced mix of positive and negative sentiments.

The FREuD Methodology

The FREuD pipeline transforms review selection into a diversification task through four key stages:

  1. Feature Extraction: Using Latent Dirichlet Allocation (LDA), the system identifies latent topics within the corpus. By focusing on nouns, these topics effectively map to specific product features (e.g., "lens," "sensor," "zoom").
  2. Sentiment Estimation: Rather than relying on coarse star ratings, FREuD utilizes the ANEW (Affective Norms for English Words) dictionary. This allows for a multi-dimensional scoring of words based on valence (positivity), arousal (intensity), and dominance (certainty).
  3. Feature-Sentiment Mapping: Topic probabilities from LDA are combined with ANEW scores to estimate the sentiment expressed toward specific features within a single review.
  4. Greedy Selection: To solve the NP-hard problem of optimal subset selection, a greedy algorithm iteratively selects reviews that offer the highest marginal gain in feature and sentiment coverage.

Experimental Evaluation

To evaluate FREuD, we crawled 2,445 reviews from CNET across three categories: Cell Phones, Printers, and Digital Cameras. Since no benchmark for review diversification existed, we established a new gold standard by having 179 human assessors manually annotate feature-level sentiments for 500 reviews.

Performance was measured using $α$-nDCG, a standard metric for evaluating intent coverage in information retrieval. We compared FREuD against two baselines: CNET-default (helpfulness ranking) and CNET-diversified (selecting reviews across different star ratings).

Key Results and Discussion

Our results indicate that FREuD consistently provides a more informative overview than existing methods.

Comparison of FREuD variants against CNET baselines using the α-nDCG@5 metric.

As shown in the figure above, the version of our algorithm that does not normalize for review length (FREuD-noLN) achieved the highest performance. Standard length normalization often favors short, sentiment-dense reviews that lack substantive detail. FREuD-noLN showed a 10.5% improvement over the CNET-default ranking and a 30% improvement over star-based diversification.

Performance of FREuD-noLN across different product categories, demonstrating consistent superiority over baselines.

Furthermore, these gains were consistent across all tested product categories, as illustrated in the category-specific breakdown. This suggests that the combination of unsupervised topic modeling and lexicon-based sentiment analysis is robust across different domains.

Conclusion and Future Work

FREuD offers a significant advancement in managing social content overload by moving beyond simple helpfulness metrics. By capturing the nuance of feature-specific sentiments, it provides users with a balanced and comprehensive view of products. Future research will focus on refining sentiment analysis to a more granular sentence level and exploring more sophisticated optimization strategies beyond greedy selection.

Summary updated

Abstract

Online discussions, user reviews and comments onthe Social Web are valuable sources of informationabout products, services, or shared contents. The rapidlygrowing popularity and activity of Web communitiesraises novel questions of appropriate aggregation anddiversification of such social contents. In many cases,users are interested in gaining an extensive overviewover pros and cons of a particular track of contributions.We address the problem of social content diversificationby combining latent semantic analysis with featurecentricsentiment analysis. Our FREuD approach providesa representative overview of sub-topics and aspectsof discussions, characteristic user sentiments underdifferent aspects, and reasons expressed by differentopponents. In experiments with real world productreviews we compare FREuD to the typical implementationof ranking reviews by the usefulness rating providedby users as well as a naive sentiment diversificationalgorithms based on star ratings. To this end we hadhuman users provide a fine-grained gold standard aboutthe coverage of features and sentiments in reviews forseveral products in three categories. We observed thatFREuD clearly outperforms the baseline algorithms ingenerating a sentiment-diversified set of user reviewsfor a given product.
BibTeX
@inproceedings{naveed2021feature,
  author = {Nasir Naveed and Thomas Gottron and Steffen Staab},
  title = {{Feature Sentiment Diversification of User Generated Reviews: The FREuD Approach}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.1609/icwsm.v7i1.14397},
}
Figure