ReviewCollage: a mobile interface for direct comparison using online reviews

Haojian Jin, Tetsuya Sakai, Koji YataniMobile HCI 2014 · DOI

TL;DR

This paper introduces ReviewCollage, a mobile interface designed to help users directly compare two online entities (like restaurants or products) by intelligently summarizing and visualizing review data. It addresses the challenge of comparing information scattered across multiple webpages on small mobile screens, offering a faster and more intuitive comparison experience than existing methods.

What Problem Does This Paper Address?

When making decisions, people often engage in a two-stage process: screening to narrow down options and then comparing the remaining candidates. While many websites effectively support screening (e.g., through filters), comparing multiple entities based on user-generated reviews remains a significant challenge, especially on mobile devices with limited screen real estate. Users often have to manually switch between different webpages, making it difficult to keep track of details and compare aspects across entities. Existing summarization tools, while helpful for understanding a single entity, often fail to facilitate direct comparison by not explicitly highlighting similarities and differences between multiple entities. This paper aims to bridge this gap by providing a unified, one-page view for direct comparison of review data.

Methods

The core of ReviewCollage lies in its novel approach to extracting and presenting review information. The system first extracts attribute-value pairs from review text, similar to prior work like RevMiner. The key innovation is the development of a Shared-Distinct value (SD-value) algorithm. This algorithm quantifies how frequently a word pair (e.g., "spicy ramen") appears in the reviews of two entities. It assigns a signed value: positive values indicate terms primarily used for one entity (distinct terms), while negative values indicate terms used for both (shared terms). This allows ReviewCollage to not only identify relevant terms but also categorize them as shared or distinct. The interface then uses a tag-cloud visualization to display these shared and distinct terms, grouped by attribute, allowing users to quickly grasp similarities and differences. A "Detail View" provides further context, including images, rating distributions, and example sentences associated with a selected word pair.

Datasets

The authors crawled reviews for 48 restaurants from Yelp.com for their word pair extraction, resulting in approximately 780,000 sentences. For the user study, they collected reviews for 16 restaurants from Yelp.com, specifically choosing categories like American, Barbeque, Chinese, and Japanese to ensure sufficient review data and allow for meaningful comparisons.

Key Insights and Ideas

  • The power of "shared" vs. "distinct" terms: Explicitly highlighting what's common and what's different between entities is crucial for efficient comparison, especially when entities are very similar.
  • Mobile-first design matters: The effort of switching between pages on a mobile device significantly hinders comparison. A single-page, aggregated view is a substantial improvement.
  • Visual cues are key: Tag clouds, when organized effectively (grouped by attribute), can quickly convey complex information about similarities and differences.
  • Pictures enhance understanding: While not always perfectly matched, users appreciate seeing images associated with reviews, even if some don't directly relate to the specific word pair.
  • User preferences vary: While distinct terms were often more useful for direct comparison, shared terms were still valuable for confirming positive or negative aspects across entities.

Key Results

In a comparative user study involving 12 participants, ReviewCollage demonstrated comparable task completion times to Review Spotlight (an existing tag-cloud interface) and was significantly faster than RevMiner. Participants using ReviewCollage were able to make decisions in an average of 105 seconds, compared to 100.2 seconds for Review Spotlight and 146.6 seconds for RevMiner (when including distractor tasks). Qualitative feedback indicated that ReviewCollage was particularly useful when comparing very similar entities, where it could effectively highlight subtle differences that other interfaces missed. Participants also found ReviewCollage's attribute-based grouping helpful for focusing on specific aspects of interest.

Limitations and Future Work

The authors acknowledge several limitations. The user study design did not assess satisfaction with decisions made in real-world scenarios, and the restaurant selection aimed to minimize prior user experience. The sample size of 12 participants was relatively small, which might have masked some quantitative differences. They also note that their summarization approach is not perfect and that future work could integrate more advanced NLP techniques. For future directions, they suggest exploring different visualization methods (like Word Spectrum) and extending ReviewCollage to support comparisons of more than two entities, which presents a significant interface design challenge. They also propose further investigation into various ranking algorithms for word pairs.

Summary updated

BibTeX
@inproceedings{jin2014reviewcollage,
  author = {Haojian Jin and Tetsuya Sakai and Koji Yatani},
  title = {{ReviewCollage: a mobile interface for direct comparison using online reviews}},
  booktitle = {Mobile HCI},
  year = {2014},
  doi = {10.1145/2628363.2628373},
}
Figure