TL;DR
This paper introduces a novel approach to detect review spammers by exploiting the "burstiness" of reviews – sudden spikes in review activity. They model reviewers and their co-occurrences within these bursts using a Markov Random Field and Loopy Belief Propagation, achieving better detection accuracy than existing methods.
What Problem Does This Paper Address?
Online product reviews are crucial for consumer decisions, but they are vulnerable to manipulation by "review spammers." These spammers post fake reviews to artificially boost or damage product reputations for profit or other gains. Existing methods often analyze reviews or reviewers in isolation. This paper addresses the challenge of identifying these spammers by recognizing that both genuine product popularity and spam attacks can lead to concentrated bursts of reviews. The core idea is that reviews within a burst are likely to share a common nature – either mostly genuine or mostly spam.
Methods
The paper's core innovation lies in exploiting review "burstiness." They first identify these bursts using Kernel Density Estimation (KDE) on review timestamps. Once bursts are identified, they construct a graph where reviewers within the same burst are connected. This reviewer network is then modeled as a Markov Random Field (MRF). The authors employ a feature-induced message passing approach within the Loopy Belief Propagation (LBP) algorithm to infer the identity (spammer, non-spammer, or mixed) of each reviewer. Key features used include:
- Ratio of Amazon Verified Purchase (RAVP): Spammers are less likely to have verified purchases.
- Rating Deviation (RD): Spammers might exhibit unusual rating patterns compared to other reviewers.
- Review Content Similarity (RCS): Spammers might reuse similar language across reviews.
- Reviewer Burstiness (RB): If a reviewer's own reviews are concentrated within product bursts, it's suspicious.
- Overall Spamming Indicator (OSI): An aggregate of the above features.
They also propose a novel evaluation method using supervised text classification, treating reviews from detected spammers as positive and non-spammers as negative.
Datasets
The study uses product reviews from the software category of Amazon.com. This dataset comprises 210,761 reviews, 50,704 reviewers, and 112,953 products. The authors mention that this data was originally crawled by the authors of Jindal and Liu (2008). No new benchmarks were introduced; instead, they leverage existing data for evaluation.
Key Insights and Ideas
- Burstiness as a Signal: The fundamental insight is that review bursts are not just random noise but can be strong indicators of either genuine popularity or coordinated spam activity.
- Networked Spammer Behavior: Spammers often collaborate. By modeling reviewers within bursts as a network, the paper captures this collaborative aspect, which isolated review analysis misses.
- Feature-Induced Message Passing: Combining behavioral features with the LBP framework allows for a more nuanced inference of reviewer identity, leveraging both local observations and network structure.
- Novel Evaluation Strategy: Using text classification on reviews as an evaluation metric, independent of the spammer detection features, provides a robust way to assess the effectiveness of the spammer identification.
Key Results
The proposed method, LBP with prior and local observation, significantly outperforms baseline methods like k-means clustering and LBP with state prior only.
- Supervised Classification: The proposed method achieved an accuracy of 71.2% for all reviews and 77.6% for burst reviews, a substantial improvement over k-means (around 53-57% accuracy) and LBP with state prior only (around 57-59% accuracy).
- Human Evaluation: Human judges showed higher agreement and better performance in identifying spammers and non-spammers when using the proposed method's output compared to k-means. The LBP with local observation achieved the best results among human evaluations.
Limitations and Future Work
- Mixed Identity: The paper acknowledges that some reviewers might exhibit both spamming and genuine behavior ("mixed" identity). While they model this, distinguishing perfectly between these states remains challenging.
- Singleton Reviewers: The current approach focuses on reviewers with multiple reviews. The authors note that detecting spammers who write only a single review is a separate challenge and a direction for future work.
- General Framework: Extending the framework to handle both reviewers with multiple reviews and single-review spammers is identified as future research.
- Dataset Specificity: While the method is applied to Amazon reviews, its generalizability to other review platforms is mentioned as a potential area for further exploration.
Summary updated