Web Item Reviewing Made Easy By Leveraging Available User Feedback

Azade Nazi, Mahashweta Das, Gautam DasarXiv 2016 · OpenReview

TL;DR

This paper introduces "TagAdvisor," a system designed to simplify the process of writing online reviews by suggesting relevant, sentiment-aware tags. It addresses the common issue of users being reluctant to write detailed reviews by providing them with pre-selected phrases that cover various aspects of a product and reflect user sentiment, making the review process faster and more engaging.

What Problem Does This Paper Address?

The proliferation of online review sites has made user feedback a crucial factor for businesses. While aggregate ratings are common, detailed reviews are essential for complex purchases. However, writing detailed reviews is time-consuming, leading to a scarcity of useful feedback. Furthermore, existing reviews can be redundant, inaccurate, or misleading. This paper tackles the challenge of encouraging "lurkers" (users who read reviews but don't contribute) to participate by making the review-writing process significantly easier. The core problem is how to intelligently suggest a concise set of tags that accurately represent an item, capture user sentiment, and cover its diverse attributes, thereby simplifying the user's task of providing feedback.

Methods

The authors propose a novel "TagAdvisor" (TA) problem, framed as a general constrained optimization problem. The core idea is to identify the top-$k$ meaningful tags for a given user and item. This is achieved by considering three key measures:

  • Relevance: How well a tag describes the item. This is modeled using rule-based classifiers to predict the probability of a tag given an item.
  • Coverage: How well the selected tags cover the different aspects (attributes) of an item. Two distinct definitions of coverage are introduced:
  • Independent Coverage (IC-TA): An attribute value is considered covered if any tag in the selected set covers it, regardless of sentiment.
  • Dependent Coverage (DC-TA): An attribute value is considered covered if it's covered by both positive and negative tags, or by one type if the other is absent. This allows for nuanced feedback on the same attribute.
  • Polarity: The sentiment distribution of the selected tags. This is influenced by a "user factor" that reflects a user's general tendency towards positive or negative feedback.

The paper formulates two specific problem instances (IC-TA and DC-TA) based on these coverage definitions. Both problems are shown to be NP-complete, leading the authors to develop exact algorithms (brute-force and Integer Linear Programming) and efficient approximation algorithms (A-IC-TA and A-DC-TA) with theoretical bounds.

Datasets

The paper utilizes both synthetic and real-world datasets for evaluation:

  • Synthetic Dataset: A large boolean matrix of 1 million items x 200 attributes (100 attributes + 50 positive tags + 50 negative tags) was generated.
  • Real Camera Dataset: Crawled from Walmart, this dataset includes 12,600 reviews from 11,500 users on 140 cameras. Item attributes were augmented by looking up Google Products and parsing 120 attributes. Tags were extracted using the AlchemyAPI.
  • Real Car Dataset: Crawled from Yahoo! Autos, this dataset contains 2350 reviews (pros and cons) for 100 used cars from 2010. It includes 47 attributes and 20 representative positive and negative tags.

Key Insights and Ideas

  • Bridging the "Lurker" Gap: The core insight is that simplifying the review process by offering pre-defined, meaningful tags can effectively engage users who are otherwise hesitant to write lengthy reviews.
  • Dual Coverage for Nuance: The distinction between Independent and Dependent Coverage is crucial. It acknowledges that users might want to express both positive and negative sentiments about the same product feature (e.g., "great camera quality, but poor battery life"), which is captured by the Dependent Coverage model.
  • User Factor for Personalization: Incorporating a "user factor" to model individual sentiment tendencies allows for personalized tag suggestions, making the system more relevant to each user.
  • NP-Completeness and Approximation: The inherent complexity of the problem (NP-complete) necessitates the development of efficient approximation algorithms, which the authors successfully demonstrate can achieve near-optimal results with significantly reduced computation time.

Key Results

  • Efficiency: The proposed approximation algorithms (A-IC-TA and A-DC-TA) are orders of magnitude faster than exact algorithms (E-IC-TA, ILP-IC-TA, E-DC-TA, ILP-DC-TA), especially as the number of tags ($k$) increases.
  • Approximation Ratio: The approximation algorithms achieve a ratio of less than 2 compared to optimal solutions, demonstrating their effectiveness.
  • Quality: Despite significant speedups, the quality of results (measured by the proportion of covered item attributes) from the approximation algorithms is comparable to that of the exact algorithms.
  • User Study: An Amazon Mechanical Turk user study revealed that 83% of users would submit reviews more often if provided with meaningful keywords. 71% of users preferred using TagAdvisor-generated tags over writing from scratch, and 77% submitted feedback that matched the suggested tags.

Limitations and Future Work

  • Scalability of Exact Algorithms: The exact algorithms (brute-force and ILP) are computationally prohibitive for larger problem instances, highlighting the necessity of approximation algorithms.
  • Submodularity of DC-TA: The Dependent Coverage function (VDC) is proven to be non-submodular, meaning standard greedy approaches don't guarantee optimal solutions, requiring specialized algorithms.
  • User Study Limitations: While the user study showed positive results, it was conducted on AMT workers, and the feedback on camera features primarily focused on external aspects rather than in-depth technical performance.
  • Future Work: The authors suggest further research into more sophisticated rule learning techniques and exploring how to leverage user preferences and feedback from similar items more effectively. They also aim to develop algorithms that can handle larger datasets and more complex scenarios.

Summary updated

Abstract

The widespread use of online review sites over the past decade has motivated businesses of all types to possess an expansive arsenal of user feedback to mark their reputation. Though a significant proportion of purchasing decisions are driven by average rating, detailed reviews are critical for activities like buying expensive digital SLR camera. Since writing a detailed review for an item is usually time-consuming, the number of reviews available in the Web is far from many. Given a user and an item our goal is to identify the top-$k$ meaningful phrases/tags to help her review the item easily. We propose general-constrained optimization framework based on three measures - relevance (how well the result set of tags describes an item), coverage (how well the result set of tags covers the different aspects of an item), and polarity (how well sentiment is attached to the result set of tags). By adopting different definitions of coverage, we identify two concrete problem instances that enable a wide range of real-world scenarios. We develop practical algorithms with theoretical bounds to solve these problems efficiently. We conduct experiments on synthetic and real data crawled from the web to validate the effectiveness of our solutions.
BibTeX
@misc{nazi2016web,
  author = {Azade Nazi and Mahashweta Das and Gautam Das},
  title = {{Web Item Reviewing Made Easy By Leveraging Available User Feedback}},
  year = {2016},
  eprint = {1602.06454},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1602.06454},
}
Figure