Exploiting Burstiness in Reviews for Review Spammer Detection

Geli Fei, Arjun Mukherjee, Bing Liu, Meichun Hsu, Malú Castellanos, Riddhiman Ghosh

TL;DR

This paper introduces a novel approach to detect review spammers by exploiting the "burstiness" of reviews – sudden spikes in review activity. They model reviewers and their co-occurrences within these bursts using a Markov Random Field and Loopy Belief Propagation, achieving better detection accuracy than existing methods.

What Problem Does This Paper Address?

Online product reviews are crucial for consumer decisions, but they are vulnerable to manipulation by "review spammers." These spammers post fake reviews to artificially boost or damage product reputations for profit or other gains. Existing methods often analyze reviews or reviewers in isolation. This paper addresses the challenge of identifying these spammers by recognizing that both genuine product popularity and spam attacks can lead to concentrated bursts of reviews. The core idea is that reviews within a burst are likely to share a common nature – either mostly genuine or mostly spam.

Methods

The paper's core innovation lies in exploiting review "burstiness." They first identify these bursts using Kernel Density Estimation (KDE) on review timestamps. Once bursts are identified, they construct a graph where reviewers within the same burst are connected. This reviewer network is then modeled as a Markov Random Field (MRF). The authors employ a feature-induced message passing approach within the Loopy Belief Propagation (LBP) algorithm to infer the identity (spammer, non-spammer, or mixed) of each reviewer. Key features used include:

  • Ratio of Amazon Verified Purchase (RAVP): Spammers are less likely to have verified purchases.
  • Rating Deviation (RD): Spammers might exhibit unusual rating patterns compared to other reviewers.
  • Review Content Similarity (RCS): Spammers might reuse similar language across reviews.
  • Reviewer Burstiness (RB): If a reviewer's own reviews are concentrated within product bursts, it's suspicious.
  • Overall Spamming Indicator (OSI): An aggregate of the above features.

They also propose a novel evaluation method using supervised text classification, treating reviews from detected spammers as positive and non-spammers as negative.

Datasets

The study uses product reviews from the software category of Amazon.com. This dataset comprises 210,761 reviews, 50,704 reviewers, and 112,953 products. The authors mention that this data was originally crawled by the authors of Jindal and Liu (2008). No new benchmarks were introduced; instead, they leverage existing data for evaluation.

Key Insights and Ideas

  • Burstiness as a Signal: The fundamental insight is that review bursts are not just random noise but can be strong indicators of either genuine popularity or coordinated spam activity.
  • Networked Spammer Behavior: Spammers often collaborate. By modeling reviewers within bursts as a network, the paper captures this collaborative aspect, which isolated review analysis misses.
  • Feature-Induced Message Passing: Combining behavioral features with the LBP framework allows for a more nuanced inference of reviewer identity, leveraging both local observations and network structure.
  • Novel Evaluation Strategy: Using text classification on reviews as an evaluation metric, independent of the spammer detection features, provides a robust way to assess the effectiveness of the spammer identification.

Key Results

The proposed method, LBP with prior and local observation, significantly outperforms baseline methods like k-means clustering and LBP with state prior only.

  • Supervised Classification: The proposed method achieved an accuracy of 71.2% for all reviews and 77.6% for burst reviews, a substantial improvement over k-means (around 53-57% accuracy) and LBP with state prior only (around 57-59% accuracy).
  • Human Evaluation: Human judges showed higher agreement and better performance in identifying spammers and non-spammers when using the proposed method's output compared to k-means. The LBP with local observation achieved the best results among human evaluations.

Limitations and Future Work

  • Mixed Identity: The paper acknowledges that some reviewers might exhibit both spamming and genuine behavior ("mixed" identity). While they model this, distinguishing perfectly between these states remains challenging.
  • Singleton Reviewers: The current approach focuses on reviewers with multiple reviews. The authors note that detecting spammers who write only a single review is a separate challenge and a direction for future work.
  • General Framework: Extending the framework to handle both reviewers with multiple reviews and single-review spammers is identified as future research.
  • Dataset Specificity: While the method is applied to Amazon reviews, its generalizability to other review platforms is mentioned as a potential area for further exploration.

Summary updated

Abstract

Online product reviews have become an important source of user opinions. Due to profit or fame, imposters have been writing deceptive or fake reviews to promote and/or to demote some target products or services. Such imposters are called review spammers. In the past few years, several approaches have been proposed to deal with the problem. In this work, we take a different approach, which exploits the burstiness nature of reviews to identify review spammers. Bursts of reviews can be either due to sudden popularity of products or spam attacks. Reviewers and reviews appearing in a burst are often related in the sense that spammers tend to work with other spammers and genuine reviewers tend to appear together with other genuine reviewers. This paves the way for us to build a network of reviewers appearing in different bursts. We then model reviewers and their co-occurrence in bursts as a Markov Random Field (MRF), and employ the Loopy Belief Propagation (LBP) method to infer whether a reviewer is a spammer or not in the graph. We also propose several features and employ feature induced message passing in the LBP framework for network inference. We further propose a novel evaluation method to evaluate the detected spammers automatically using supervised classification of their reviews. Additionally, we employ domain experts to perform a human evaluation of the identified spammers and non-spammers. Both the classification result and human evaluation result show that the proposed method outperforms strong baselines, which demonstrate the effectiveness of the method.
BibTeX
@inproceedings{fei2021exploiting,
  author = {Geli Fei and Arjun Mukherjee and Bing Liu and Meichun Hsu and Malú Castellanos and Riddhiman Ghosh},
  title = {{Exploiting Burstiness in Reviews for Review Spammer Detection}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.1609/icwsm.v7i1.14400},
}
Figure