An Algorithm Model For Incremental Dectection of Spam Reviews

Maoan Wang, Jun Sun, Yifan Wu, Guoshi WuInternational Journal of Modeling and Optimization 2016 · DOI

Abstract

Surging smartphone use and pervasive O2O services mean customers can post their reviews about restaurants and shops online. However, many merchants may hire some people to post positive but fraud reviews in order to attract more customers. Therefore, a model need to be built to detect spam reviews. In this paper, firstly, we build a detection model using traditional batch processing which view the detection as a binary classification problem. Next, since many reviews are coming sequentially, batch processing is not efficient and useful. We will use another incremental algorithm-Hoeffding Option Tree to update the model without processing the past data repeatedly. We find that the incremental method can drastically improve the speed and the accuracy is also satisfying.

Introduction

In the ecosystem of online-to-offline (O2O) services like Yelp or Dianping, fraudulent positive reviews—often termed "hyper spam"—pose a significant threat to platform integrity. Merchants frequently hire individuals to post deceptive reviews to artificially boost their ratings.

The primary technical hurdle in combating this is the high velocity of incoming data. Traditional "batch processing" models require retraining on the entire historical dataset whenever new reviews arrive, a process that becomes prohibitively expensive and slow as the platform scales. This paper proposes a transition to incremental learning, specifically utilizing the Hoeffding Option Tree, to detect spam sequentially and update the model "on the fly."

Methodology

The proposed detection framework operates in two distinct phases: credibility assessment and incremental classification.

Credibility Modeling

Before classification, the system establishes a foundation by calculating credibility scores for both users and merchants using Logistic Regression.

  • User Credibility: Derived from features such as rating deviation (the difference between a user's score and the average) and the "Early Time Frame" (identifying users who post immediately after a merchant's debut).
  • Merchant Credibility: Assessed through metrics like the ratio of single-review users to the total review count.
Architecture of the spam review detection framework.

Incremental Classification with Hoeffding Option Trees

The core of the system is the Hoeffding Option Tree (HOT). Unlike standard decision trees that require a full dataset for training, the HOT uses the "Hoeffding Bound" to determine the minimum number of observations needed to make a statistically significant split at a node.

As an ensemble method, it allows for "option nodes," enabling the model to explore multiple potential paths for a single data point. This makes the model significantly more robust than a simple Hoeffding Tree while maintaining the efficiency of incremental updates.

Flowchart of the Hoeffding Option Tree incremental learning process.

Data and Features

The study utilized a dataset from Dianping.com, comprising 331,415 reviews. Key features included User ID, shop ID, review body, star rating, and review type. Preprocessing involved using the Ansj Chinese segmentation tool to extract linguistic features, such as the distribution of parts of speech (adjectives, verbs, nouns) and review length.

Results and Discussion

The performance of the incremental Hoeffding Option Tree was compared against a static Random Forest baseline (200 trees).

  • Accuracy: The static model achieved an average accuracy of 75.33%. In contrast, the incremental model reached between 80.2% and 82.6%, depending on the update frequency ($n_{min}$). This suggests that Option Trees can outperform traditional "bagging" methods in streaming environments.
  • Efficiency: The static model required approximately 2,847 seconds (47 minutes) for processing. The incremental model updated in just 2.7 to 4.5 seconds, representing a massive leap in operational efficiency.
  • Scalability: The incremental model maintained high accuracy even with smaller tree sizes, making it highly memory-efficient for long-term deployment.

A notable observation is the "credibility bottleneck." While the classifier is incremental, the global credibility scores still require periodic full recalculations, highlighting a challenge in creating a "fully" incremental system.

Conclusion

This research demonstrates that incremental learning via Hoeffding Option Trees offers a superior alternative to batch processing for real-time spam detection. Future work will focus on making the credibility assessment phase fully incremental and exploring how feature sets can evolve to counter adaptive spamming tactics.

Summary updated

Abstract

Surging smartphone use and pervasive O2O services mean customers can post their reviews about restaurants and shops online. However, many merchants may hire some people to post positive but fraud reviews in order to attract more customers. Therefore, a model need to be built to detect spam reviews. In this paper, firstly, we build a detection model using traditional batch processing which view the detection as a binary classification problem. Next, since many reviews are coming sequentially, batch processing is not efficient and useful. We will use another incremental algorithm-Hoeffding Option Tree to update the model without processing the past data repeatedly. We find that the incremental method can drastically improve the speed and the accuracy is also satisfying.
BibTeX
@article{wang2016algorithm,
  author = {Maoan Wang and Jun Sun and Yifan Wu and Guoshi Wu},
  title = {{An Algorithm Model For Incremental Dectection of Spam Reviews}},
  journal = {International Journal of Modeling and Optimization},
  year = {2016},
  doi = {10.7763/ijmo.2016.v6.501},
}
Figure