Abstract
Online reviews have increasingly become a very important resource for consumers when making purchases. Though it is becoming more and more difficult for people to make well-informed buying decisions without being deceived by fake reviews. Prior works on the opinion spam problem mostly considered classifying fake reviews using behavioral user patterns. They focused on prolific users who write more than a couple of reviews, discarding one-time reviewers. The number of singleton reviewers however is expected to be high for many review websites. While behavioral patterns are effective when dealing with elite users, for one-time reviewers, the review text needs to be exploited. In this paper we tackle the problem of detecting fake reviews written by the same person using multiple names, posting each review under a different name. We propose two methods to detect similar reviews and show the results generally outperform the vectorial similarity measures used in prior works. The first method extends the semantic similarity between words to the reviews level. The second method is based on topic modeling and exploits the similarity of the reviews topic distributions using two models: bag-of-words and bag-of-opinion-phrases. The experiments were conducted on reviews from three different datasets: Yelp (57K reviews), Trustpilot (9K reviews) and Ott dataset (800 reviews).
Introduction
Online reviews are a cornerstone of modern commerce, but they are plagued by "opinion spam." While researchers have become adept at catching "elite" spammers—prolific users with suspicious posting patterns—a massive blind spot remains: the singleton. Studies show that over 90% of reviewers on some platforms only ever post once. If a company hires a "click farm" to write hundreds of fake reviews, and each review is posted under a unique, one-time-use name, traditional behavioral filters fail. There is no history to analyze.
To catch these "ghosts," we must look at the only thing they leave behind: the text itself. Most review spam detection relies on "behavioral footprints"—tracking users who post too often or from the same IP. This paper tackles the much harder problem of singleton spammers: one-time posters who leave no trail. By leveraging semantic similarity and topic modeling, the authors show that we can catch these fraudsters simply by identifying the "limited imagination" they show when rephrasing the same fake sentiments across different accounts.
Methodology
The authors propose two primary text-based approaches to identify reviews that are semantically identical despite using different words:
- Knowledge-Based Semantic Similarity: Instead of just counting shared words, this method uses WordNet (a lexical database) to understand synonymy. It calculates a weighted similarity score between two documents by looking at how closely the nouns, verbs, and adjectives in one review match the "synsets" (synonym groups) of the other.
- Topic Modeling (LDA): The authors used Latent Dirichlet Allocation to see if two reviews share the same underlying "topic distribution." They tested two versions: Bag-of-Words and Bag-of-Opinion-Phrases. To compare these distributions, they used the Jensen-Shannon (JS) divergence, which measures how much two probability distributions overlap.
Experimental Setup
The study evaluated these methods across three distinct datasets to ensure the results weren't just a fluke of one platform's quirks:
- Yelp: 57,000 reviews from NYC restaurants, using Yelp’s own "filtered" vs. "recommended" status as the ground truth for spam.
- Trustpilot: 9,000 labeled English reviews provided directly by the company, specifically focusing on one-time reviewers.
- Ott Dataset: A "gold standard" benchmark of 800 reviews created via crowdsourcing, where participants were paid to write deceptive hotel reviews.
Results and Discussion
The core finding is the "Imagination Gap": spammers are prone to rephrasing rather than inventing. They might swap "excellent meal" for "great food," but the semantic core remains identical. Semantic similarity captures this "subtle textual clue" where basic math (cosine similarity) sees two different sentences.
Performance on Yelp
The semantic similarity method consistently outperformed the vectorial baselines. On the Yelp dataset, it achieved 90% precision at a similarity threshold of 0.8.
Performance on Trustpilot
The results were even stronger on Trustpilot, reaching over 90% precision at a 0.85 threshold. The authors suggest this is because Trustpilot spammers are less skilled at hiding their tracks or that Yelp spammers are more "professional" and work harder to bypass existing filters.
While LDA (specifically the 30-topic model) showed a "smooth climb" in precision as the threshold increased, it generally achieved lower overall precision than the semantic similarity approach. In the Ott dataset, the authors showed a clear, measurable gap between the similarity distributions of truthful reviews versus fake ones, proving that deceptive writing is fundamentally more repetitive.
Conclusion
The main trade-off identified is Recall. While the models are highly precise (when they flag something as spam, they are usually right), the recall drops significantly at high thresholds. This means they catch the "lazy" spammers but might miss the more creative ones. Future production-ready systems should likely combine these semantic features with sparse behavioral data to create a more robust defense.
Summary updated