ScoreGAN: A Fraud Review Detector Based on Regulated GAN With Data Augmentation

Saeedreza Shehnepoor, Roberto Togneri, Wei Liu, Mohammed BennamounIEEE Trans. IFS 2022 · DOI

TL;DR

This paper introduces ScoreGAN, a novel Generative Adversarial Network (GAN) designed to combat fraudulent online reviews. By incorporating review scores into the GAN's generation process and employing Information Gain Maximization, ScoreGAN effectively generates more realistic and score-correlated fake reviews, significantly improving the detection of both human-written and bot-generated fraud.

What Problem Does This Paper Address?

The proliferation of online reviews is a double-edged sword. While they offer valuable insights for consumers, they also present a fertile ground for manipulation. Marketers and malicious actors can deploy "fraudulent reviews" – fake opinions designed to artificially boost or damage a business's reputation. Detecting these reviews is challenging due to the scarcity of high-quality labeled data and the increasing sophistication of fraud techniques, including the use of bots that can mimic human writing styles and manipulate metadata. Existing methods often struggle to incorporate crucial contextual information like review scores and are vulnerable to bot-generated content that leaves minimal traces of human activity.

Methods

ScoreGAN tackles this problem by building upon the Generative Adversarial Network (GAN) framework. The core innovation lies in its ability to generate score-correlated reviews. This is achieved through a novel application of Information Gain Maximization (IGM) within the GAN's objective function.

Here's a breakdown of the key components:

  • Generator (Module 1): This component is responsible for creating synthetic fraud reviews. Unlike standard GANs, ScoreGAN's generator takes both random noise and the review score as input, ensuring that the generated reviews are semantically aligned with their assigned scores. This is crucial for mimicking human review behavior, where sentiment (reflected in the score) is a key aspect.
  • Discriminators (Module 2 & 3): ScoreGAN employs two discriminators:
  • $D_g$: This discriminator's primary role is to distinguish between genuine and fraud reviews.
  • $D_f$: This discriminator is designed to differentiate between human-written fraud reviews and bot-generated fraud reviews. It also incorporates an auxiliary function, driven by IGM, to calculate the probability of a review's score based on its features. This dual role helps in generating more human-like bot reviews.
  • Information Gain Maximization (IGM) Regularization: The paper leverages IGM to maximize the mutual information between the review score and the generated review. This regularization term is added to the GAN's objective function. It helps stabilize the GAN's training process, addressing a common limitation of GANs, and guides the generator to produce reviews that are not only fake but also semantically consistent with their scores.
  • Joint Representation: ScoreGAN learns a joint representation of review text (using GloVe embeddings) and review scores. This combined representation is fed to the discriminators, allowing them to better capture the nuances of fraudulent reviews, especially those generated by bots.

Datasets

The paper evaluates ScoreGAN on two real-world datasets:

  • Yelp-main: This dataset contains reviews for restaurants and hotels, with scores ranging from 1 to 5. It's noted for its relatively trusted Yelp filtering system for labeling spam.
  • TripAdvisor: This dataset includes reviews for entertainment places, with a binary "like" or "dislike" sentiment label.

Key Insights and Ideas

  • Scores Matter: The paper strongly emphasizes that review scores are a critical piece of information for detecting fraud. Simply using text alone is insufficient, and incorporating scores into both generation and detection significantly boosts performance.
  • Generated Data Augmentation is Powerful: By generating realistic, score-correlated fake reviews, ScoreGAN effectively augments the limited real fraud data. This augmented dataset allows the discriminators to learn more robustly and generalize better, especially against sophisticated bot-generated content.
  • Simplicity Can Be Effective: While other methods might try to incorporate numerous behavioral features, ScoreGAN demonstrates that focusing on the score, combined with text, can be highly effective and avoid the computational cost and potential noise of too many features.
  • GAN Stability is Achievable: The proposed IGM-based regularization technique helps stabilize GAN training, a notorious challenge in the field, leading to faster convergence and more reliable results.
  • Bot Detection is Enhanced: By explicitly training the generator to produce human-like bot reviews and using a dedicated discriminator ($D_f$), ScoreGAN shows improved capability in identifying bot-generated fraud, which often evades traditional methods.

Key Results

ScoreGAN demonstrated significant improvements over existing state-of-the-art methods on both the Yelp and TripAdvisor datasets.

  • Performance Improvement: ScoreGAN achieved a 7% improvement in Average Precision (AP) on the Yelp dataset and a 5% improvement on TripAdvisor compared to FakeGAN.
  • Robustness to Data Scarcity: The model showed remarkable robustness when trained with limited data (as low as 30% supervision), maintaining competitive performance, especially compared to supervised methods whose performance degraded significantly.
  • Effect of Score: Including the score in both the generator and discriminator consistently improved performance across metrics. The generator's use of the score was found to be particularly impactful.
  • Regularization Benefits: The regularization term significantly improved GAN stability and convergence speed, requiring fewer iterations to reach optimal performance compared to an unregulated version.

Limitations and Future Work

  • Limitations:
    • The paper notes that for the TripAdvisor dataset, the binary "like/dislike" score might limit the fine-grained generation of score-correlated reviews compared to the 1-5 scale of Yelp.
  • While ScoreGAN shows robustness, the performance still generally improves with more training data, highlighting the inherent need for data in deep learning.
  • Future Work: The authors suggest extending their work to incorporate a wider range of features beyond just text and scores, aiming for a more comprehensive joint representation of various metadata. This could further enhance fraud detection capabilities.

Summary updated

BibTeX
@article{shehnepoor2022scoregan,
  author = {Saeedreza Shehnepoor and Roberto Togneri and Wei Liu and Mohammed Bennamoun},
  title = {{ScoreGAN: A Fraud Review Detector Based on Regulated GAN With Data Augmentation}},
  journal = {IEEE Trans. Inf. Forensics Secur.},
  year = {2022},
  doi = {10.1109/tifs.2021.3139771},
}
Figure