Telling Real Change from Fake Reviews with a Per-Business Fact Timeline

Fake-review detectors that watch for sudden rating swings share a blind spot: real businesses also change suddenly. A restaurant gets a new owner, an app ships a broken update, a hotel renovates. Honest customers then flood in with sharp, one-sided ratings that look exactly like a coordinated campaign. This proposal builds, for each business, a running timeline of facts about it, extracted from its own reviews. Each fact (for example, "the patio is closed" or "the app logs you out") gets the date it became true and the date it stopped being true. A fact enters the timeline only when several independent, established reviewers report it and later reviewers keep confirming it. A rating burst is then judged by a simple test. If the burst carries new facts that persist afterward, the business probably changed. If its claims vanish once the burst ends, or contradict the timeline at the time they were posted, it is probably a campaign.

The idea borrows from research on stale knowledge in language models. That work treats facts as having lifespans and shows that models fail when they cannot tell current facts from superseded ones. Here those lessons are applied to the reverse problem: an evolving record of facts is used to decide which sudden changes in opinion are real. The same timeline also targets a known weak spot of current detectors. Fake negative reviews ("smearing") are hard to catch because genuine complaints look identical to them, and this method can flag a complaint that revives a defect the timeline shows was already fixed.

Core Claim

Temporal spam detectors flag distributional and burst anomalies. One line of work does this with multivariate rating time series and change-point alarms (Temporal Opinion Spam Detection by Multivariate Indicative Signals). Another uses inter-arrival timing and co-bursting reviewer networks (Modeling Review Spam Using Temporal Patterns and Co-bursting Behaviors). A third uses departures from the normal J-shaped star distribution (Distributional Footprints of Deceptive Product Reviews). All three treat an abrupt shift as suspicious in itself. The Ye et al. system also needs a human to decide whether an alarm reflects a campaign or genuine interest. Li et al. explicitly discard review text, and they note that viral events and promotions draw organic surges that add noise to their networks.

We claim that the missing signal is whether a burst's content becomes durable truth about the business. A detector that adjudicates alarms with this signal should cut false alarms on genuine state changes substantially, without losing recall on campaigns. Durability means corroboration by reviewers outside the burst, persistence past the burst, and consistency with a timeline of facts at each review's timestamp. The same timeline should also raise accuracy on fake negative reviews above the roughly 65.5% plateau reported in Distributional Footprints of Deceptive Product Reviews.

What is new is not adding text to a spam detector. That has been done, for example by the metadata priors in Collective Opinion Spam Detection: Bridging Review Networks and Metadata. What is new is giving the detector an explicit record of which facts about a business were true when, and using fact persistence as evidence about intent.

Research Program

Building the timeline. For each business we extract short claims about aspects of it from every review, such as the chef, opening hours, a menu item, parking, or an app login bug. Each claim carries a stance and a timestamp. These claims are then merged into "state units": structured records of one aspect's value, its validity interval, and an explicit link to the unit it replaced. The design follows StateMem in Can Agent Memory Systems Track Evolving State?, which stores facts as typed units with dependency edges and resolves the current value deterministically rather than letting a language model read raw history.

That paper constrains our design as much as it supplies it. It shows that models anchor on loudly repeated older facts even when the correct update is in context, and that chain-of-thought reasoning can rationalize stale facts. Spammers repeat claims on purpose, so a timeline whose admission rule counts raw mentions would be easy to game. Our admission rule therefore counts independent sources, not mentions. A source is a reviewer with a history outside the burst, and a source in the same co-bursting cluster as another counts once.

We also follow SUMIE: A Synthetic Benchmark for Incremental Entity Summarization in two ways. We merge batches of claims (its MERGE strategy) rather than updating after each review, because MERGE outperformed one-at-a-time updating there. We also measure how often extracted attribute values are unsupported, since SUMIE's human evaluation found about 25% unsupported attribute-value pairs even with strong models.

Expected lifespans. Following Mitigating Temporal Misalignment by Discarding Outdated Facts, which predicts how long a fact will stay true and discounts confidence once that time is exceeded, we train a duration predictor per aspect type. A "closed for renovation" claim is expected to last weeks; a "new owner" claim is expected to last years. This matters because persistence must be judged against expected lifespan. A genuine temporary closure should not be penalized for disappearing.

Adjudicating bursts. For a burst $b$, let $C_b$ be the set of new claims it introduces. Let $\rho(c)$ be the fraction of claim $c$'s expected lifespan, as predicted for its aspect, over which independent reviewers posting after the burst continue to support it. The burst's explanation score is

$$E(b) = \frac{1}{|C_b|} \sum_{c \in C_b} \rho(c).$$

A high $E(b)$ means the burst told us something durable about the business. A low $E(b)$ means its claims evaporated.

Each review also receives a consistency score: how well its claims agree with the timeline at its own timestamp, built only from earlier reviews. A review that asserts a superseded or not-yet-true fact scores low. Both scores feed in as node priors to a SPEAGLE-style graph model (Collective Opinion Spam Detection: Bridging Review Networks and Metadata), a Markov random field over users, reviews and products whose beliefs are spread by loopy belief propagation. They replace part of its hand-built metadata priors.

Detection latency. Because $E(b)$ needs reviews posted after the burst, detection is deliberately delayed. We report the trade-off between delay and accuracy explicitly, as a curve over waiting windows from days to months, instead of hiding it.

Why reviews and not the web. We build the timeline from reviews rather than external search because small businesses have little up-to-date web presence. RealTime QA: What's the Answer Right Now? found that 34% of errors in its search-augmented system came from retrieval failures, and that models fall back on stale knowledge when retrieval fails. External sources such as official menus or app changelogs are used only as an optional check where they exist.

Baselines

Behavioral alarms with no adjudication. The naive reference is the lead-then-local time-series detector of Temporal Opinion Spam Detection by Multivariate Indicative Signals, which flags every alarm as spam. It is the most direct statement of the "shifts are suspicious" assumption we challenge. It is run on identical product streams with thresholds tuned on the same validation months. The file reports only qualitative findings for it (for example, an app receiving over 900 reviews a day, almost all 5-star from singleton accounts), so there is no reported quantitative result found on Lacuna for it on SWM or Flipkart. Beating it means a lower false-alarm rate on annotated genuine-change windows at matched campaign recall.

The strongest existing method. This is SPEAGLE+ from Collective Opinion Spam Detection: Bridging Review Networks and Metadata, which already combines graph structure, text-derived metadata and behavior on YelpChi, YelpNYC and YelpZip. Its reported result is relative: precision@k increasing by up to 56% in absolute terms with as little as 1% of labeled review data. We reproduce it on the same splits with the same 1% label budget and the same graph. The only change in our system is the timeline-based priors.

A text-only detector. We include a fine-tuned BERT classifier in the style of Fact or Factitious? Contextualized Opinion Spam Detection, which reports 90.5% accuracy on OpSpam. It is retrained on time-ordered splits so that its temporal degradation is visible. That paper reports no Yelp accuracy number in the file.

The temporal-farming detector. On Dianping we compare against the LHMM co-bursting model of Modeling Review Spam Using Temporal Patterns and Co-bursting Behaviors, which models "farming" and "harvesting" phases of fake accounts and reports an 85.41% detection rate for such "raised accounts."

Ablations. Two ablations isolate the key ingredient. The first, "aspect features without time," gives SPEAGLE+ the same extracted claims as bag-of-aspect priors with no validity intervals, no persistence and no independence weighting. The second, "persistence without independence," counts raw mentions. If the first ablation matches the full system, the timeline adds nothing beyond better text features. If the second matches it, the anti-salience design is unnecessary.

Why not just…?

Why not just add text or aspect features to a supervised burst classifier? This is the real threat. A classifier trained on per-window text embeddings could in principle learn that "new owner" bursts are benign. The "aspect features without time" ablation tests it directly. Our prediction is that it fails on exactly the cases that matter. A genuine change and a campaign can use the same vocabulary within the window, and what separates them is what happens afterward and who else confirms it, which a within-window classifier never sees. If the ablation comes within noise of the full system on the annotated change windows, we will drop the timeline and report that simpler result.

Why not just prompt a large language model with the business's review history and ask whether a burst is fake? Can Agent Memory Systems Track Evolving State? shows that models given complete, correct context still anchor on salient superseded facts, and that reasoning traces sometimes lower accuracy by rationalizing stale facts. Salience is exactly what a spam campaign manufactures. We nonetheless run this baseline, using a long-context model given all prior reviews with the same extraction model's access, on the annotated set. We report it separately on campaigns that repeat a claim heavily, where we expect it to fail most.

Why not just retrain the detector regularly on new reviews? TimeLMs: Diachronic Language Models from Twitter shows that quarterly updates help with drifting language, and that static models lose nearly 10% after a year. But retraining on unfiltered reviews absorbs campaign claims as truth, and it does not represent which business attribute changed when. We include a quarterly-retrained BERT baseline to measure how much of our gain is just freshness.

Why not just filter by reviewer age and singleton status? Reviewer age ("youth score") and singleton ratio are already among Ye et al.'s signals. Raised accounts are built precisely to defeat them by farming normal histories, as Li et al. describe. Our independence rule uses reviewer history too, but adds the requirement that claims survive the burst, which a farmed account cannot arrange alone.

Proposed Experiments

Data. We use YelpZip and YelpNYC, which have timestamps, text and filter labels, Dianping for raised accounts, and SWM and Flipkart for app and product bursts. Because Yelp's filter is itself proprietary and may flag genuine change bursts, filter labels alone cannot measure the false-positive problem.

A new annotated set of genuine changes. We therefore build one. We first find candidate change windows using explicit cues such as "under new management," "renovated" or "after the update." Annotators then confirm each window from the review stream and, where available, external records. A matched set of windows flagged by Yelp's filter or by co-bursting clusters serves as campaigns.

A semi-synthetic test. We also inject campaigns written by language models into real business histories. Some are prompted only with the business's old reviews, so they produce fake claims about features or defects that were already superseded. Others are given current facts, to test the method against well-informed fakes.

Metrics. We report precision@k and AUC on the Yelp labels for comparability with SPEAGLE+. On the annotated set we report false-alarm rate at fixed campaign recall. We report detection rate on Dianping raised accounts, accuracy on fake negative reviews against the 65.5% reference, a latency-versus-accuracy curve, and the unsupported-claim rate of the timeline itself.

Decisive checks. Three results decide the project. First, at matched campaign recall, the full system must cut false alarms on genuine-change windows relative to both SPEAGLE+ and the "aspect features without time" ablation. Second, on informed fakes, where injected claims match the current timeline, the gain should shrink but recall should not fall below SPEAGLE+. That would show the method degrades gracefully rather than being trivially evaded. Third, accuracy on fake negative reviews that revive superseded defects should clearly exceed 65.5%, while accuracy on fake negatives that make claims consistent with the timeline should not. That pattern would confirm that the gain comes from the timeline and not from a generic text signal.