Most systems that detect false videos treat every upload as a new puzzle. They decode the frames, run a large vision-language model, and sometimes search the web, all from scratch. The FakeSV benchmark suggests this wastes effort. It found that 39% of fake short videos appeared after the claim had already been officially debunked (FakeSV). A large share of the misinformation stream is therefore old falsehoods coming back: the same footage re-uploaded, or the same claim reworded. We propose to handle that share with a time-stamped memory of past verdicts. New videos are matched against this memory using signals read directly from the compressed video file, without decoding it to pixels. Two temporal corrections make the matches trustworthy. First, each verdict is discounted as it ages. Second, the text matcher is kept current as the wording of claims changes. Only videos that the memory cannot confidently resolve go on to an expensive verifier.
The opportunity matters because the cases this memory targets are the ones where current models are weakest. When a language model is shown a false claim repeatedly, its uncertainty drops by about 52.8% and it adopts the falsehood (Understanding Knowledge Drift in LLMs through Misinformation). A heavily recirculated rumor therefore makes the model's own confidence least reliable at the moment it is most needed. An external record of what was debunked, and when, does not suffer from this. Health claims add one more requirement. Some verdicts genuinely expire when clinical guidance changes, so a memory that never forgets would wrongly flag claims that have since become true.
Core Claim
Under a strict temporal split, a two-stage system beats running the strongest verifier on everything. The first stage is a cheap screener that works on the compressed file and matches uploads against a memory of prior verdicts. The second stage is a strong verifier that receives only the videos the screener cannot resolve. We claim this system matches that verifier's accuracy within about one point at a fraction of its compute, and degrades less as the test period moves further past the training period. The new ingredient is not compressed-domain features, recirculation, or fact lifespans separately. It is the combination: a verdict memory whose matches are discounted by predicted claim lifespan and whose text keys are refreshed by lexical trend detection.
The claim can be shown false. If only a small fraction of test-period fakes has a genuine earlier match in the memory, recirculation is not a usable structure and the proposal fails. We measure this fraction first.
What Prior Work Does and Leaves Open
Current video misinformation detectors provide the verifiers we escalate to, and they motivate the cost argument. SV-FEND fuses titles, transcripts, audio, VGG19 keyframes, C3D clips, comments and publisher profiles (FakeSV). FakeSV-VLM adapts InternVL with a mixture-of-experts adapter and an event-alignment check (FakeSV-VLM). FactGuard runs an agent trained with reinforcement learning that alternates web search and frame inspection, and it names latency as its main limitation (FactGuard). All three decode pixels for every video, and all three score each video independently of verdicts already issued. We keep them as the second stage and add the memory in front of them.
Compressed-domain methods supply the screener's visual channel. The compressed domain here means the data a video codec already stores: motion vectors (block displacements between frames), low-resolution "DC images" read from the lowest-frequency coefficients of intra-coded frames, and per-frame bit counts. Three results show these signals are useful and cheap. TCSF grounds sentences in video from these signals and runs 100 ActivityNet videos in 46.16 s against 637.51 s for MMN (You Can Ground Earlier than See). The sequence of frame sizes alone classifies genre and creator about 15,000 times faster than real time (Judging a video by its bitstream cover). DC images have long supported shot-boundary indexing (Matsumoto).
These results also set limits. Frame-size envelopes cannot separate fine-grained content (Judging a video by its bitstream cover), so we use them only as one channel for finding near-duplicate footage, never for judging truth. CVT5 found that residuals hurt captioning (CVT5), while TCSF found them useful as a gate between motion and appearance. We therefore leave residuals out by default and test them in an ablation. The misinformation surveys treat compression only as a nuisance that destroys pixel forensics (Online Misinformation Video Detection: A Survey; Combating Online Misinformation Videos). We use compressed data as a source of retrieval keys instead.
Two lines of temporal NLP supply the corrections. Zhang and Choi predict how long a fact stays true and discount confidence once the gap between query time and evidence time exceeds that duration, which cuts calibration error by 50–60% on SituatedQA (Mitigating Temporal Misalignment by Discarding Outdated Facts). We reuse their duration model, applied to stored verdicts rather than retrieved answers. Chen, Neves and Solorio score how fast a term's frequency is rising and use that score to pick retraining data, improving named-entity F1 by 2.70–5.71% over random selection (Mitigating Temporal-Drift). TimeLMs shows social-media models lose nearly 10% within a year without updates (TimeLMs). We use trend scoring to decide which memory keys to re-embed and which claim paraphrases to add, rather than to retrain a tagger.
HealthFC constrains the fallback design. Web snippets as evidence give only 44.4% macro F1, against 67.5% with curated clinical evidence (HealthFC). Search for "natural remedy" queries drifts toward unvetted forums (Hill et al.). Both findings argue for a curated verdict store over open web search.
Research Program
The memory stores one entry $m$ for each resolved item: fake videos, debunking videos, and verified-real videos available before a cutoff. Each entry holds four things: the verdict, its timestamp $t_m$, a text key built from the title, the ASR transcript and extracted claim spans, and a compressed visual key. The visual key concatenates three parts, all computed with bitstream parsing (PyAV/ffmpeg side data) and partial entropy decoding, with no full reconstruction:
- the frame-size envelope, normalized to be invariant to bitrate;
- a small embedding of the DC images of intra frames;
- pooled motion-vector statistics around scene cuts.
The visual encoder is trained contrastively so that a clip matches its own re-encoded, cropped, overlaid and speed-changed versions. These are exactly the edits applied when footage is repurposed.
For an incoming video $x$ at time $t_q$, the screener retrieves candidates and scores each one as
$$s(x,m) = \mathrm{sim}(x,m)\,\bigl(1 - F_{c}(t_q - t_m)\bigr),$$
where $\mathrm{sim}$ is a learned combination of text and visual similarity, and $F_c$ is the cumulative distribution of the predicted validity duration for the claim $c$ attached to $m$, following Zhang and Choi. A claim of the form "vaccine X causes infertility" should carry a long predicted duration. Guidance such as "masks are not recommended" should carry a short one. The screener propagates the stored verdict if $s$ exceeds a threshold tuned on a held-out earlier period. Otherwise it escalates the video to the verifier.
Between memory updates, a trend score over claim n-grams in the incoming stream flags terms that are bursting. Entries sharing those terms are re-embedded with the current text encoder and expanded with paraphrases mined from newly resolved items. Stored verdicts are never retrained into model weights. This keeps the memory non-parametric and therefore immune to the confidence collapse caused by repeated exposure.
Baselines
The naive reference is a detector that uses titles and transcripts only, trained on the pre-cutoff period. HealthFC's claims-only result (39.3% F1) shows how weak text-without-evidence can be in health. A title-only model tells us how much the rest of the system actually adds.
The strongest existing method is FakeSV-VLM, which reports 90.22% accuracy on FakeSV and 89.30% on FakeTT (FakeSV-VLM). We also run SV-FEND, which reports 79.31% accuracy and 79.24% F1 on FakeSV (FakeSV). FactGuard reports 79.3% accuracy and 81.4% F1 on FakeSV (FactGuard); we run it if its tools can be reproduced. Those numbers come from each paper's own splits. We retrain every baseline on our temporal split, using the same pre-cutoff data the memory sees, the same hyperparameter search budget, and GPU-seconds measured on the same hardware.
Beating FakeSV-VLM means one of two outcomes. The cascade matches its temporal-split accuracy within one point while using at most a third of its per-video compute. Or the cascade's accuracy falls off more slowly than FakeSV-VLM's as the test window moves one and two years past the cutoff. Matching FakeSV-VLM on accuracy alone at equal cost would not count as a win.
Four ablations isolate the key ingredients:
- The same cascade with the duration discount removed, which tests whether verdict aging matters.
- The same cascade with trend-driven refresh removed.
- The same cascade with the text channel only, which tests whether the compressed visual key earns its place.
- The same cascade with residuals added to the visual key, which settles the CVT5–TCSF disagreement for this task.
Why not just…?
Why not just decode one keyframe per second and match with CLIP or a perceptual hash? This is the most serious threat. Decoding a few intra frames is cheap, and CLIP embeddings are robust to crops and overlays. The case for the compressed key rests on screening throughput and on the envelope's ability to capture editing rhythm. It does not rest on better accuracy per match. We run exactly this alternative as a visual-key variant. If it gives higher match recall at a similar cascade cost, we will adopt it, and the contribution becomes the temporal verdict memory rather than bitstream features.
Why not just run FakeSV-VLM on every video? At 90.22% it may already be close to the ceiling on random splits. However, no temporal-split result for it appears on Lacuna, and TimeLMs' roughly 10% one-year decay suggests the ceiling moves. The degradation-over-time curve in our experiments tests this directly. If FakeSV-VLM does not decay and cost is irrelevant to the deployment, the memory offers only savings.
Why not just search the web for every claim, as FactGuard does? For health claims, HealthFC shows web snippets reach 44.4% F1 against 67.5% with curated evidence. Hill et al. show that alternative-remedy queries pull results toward forums. Search also adds network latency to every video. We measure FactGuard-style retrieval on the health slice alongside the memory.
Why not just fine-tune the detector on each new period? This is the TimeLMs recipe. It is a fair competitor, so we give it the same newly resolved items the memory receives. Retraining, however, folds verdicts into model weights. That is precisely the mechanism the knowledge-drift study shows can be hijacked by repeated falsehoods, and it cannot express that a specific verdict has expired. We compare the two under matched data and report where each wins.
Proposed Experiments and Decisive Checks
The main testbed is FakeSV, which covers 2017–2022, with a chronological cutoff: earlier items form the memory and the training data, later items are the test set. We add a health-topic subset selected by keyword and verified manually. FakeTT tests cross-platform transfer if upload dates can be recovered; otherwise we report it only as a random-split sanity check. The duration model is validated on SituatedQA before transfer, where its reported numbers are 20.5 EM with retrieval on 45% of queries. For verdict expiry we build a small probe set of health claims whose official status changed during 2020–2022, since no existing benchmark on Lacuna covers this. We report the following metrics:
- accuracy and macro F1;
- the fraction of test videos the memory resolves, and its precision on them;
- GPU-seconds per video;
- the false-flag rate on the probe set.
Three checks decide the project:
- Is recirculation common enough? We measure what share of post-cutoff fakes has a true earlier match, and stop if it is below roughly 15%.
- Does the visual key survive platform re-encoding? It must reach match recall comparable to decoded-keyframe CLIP on re-uploaded pairs.
- Does the duration discount help on changed claims? It must cut false "fake" flags on the probe set without lowering precision on stable debunks.
References
- FakeSV: A Multimodal Benchmark with Rich Social Context for Fake News Detection on Short Video Platforms
- Understanding Knowledge Drift in LLMs Through Misinformation
- FakeSV-VLM: Taming VLM for Detecting Fake Short-Video News via Progressive Mixture-Of-Experts Adapter
- FactGuard: Agentic Video Misinformation Detection via Reinforcement Learning
- You Can Ground Earlier than See: An Effective and Efficient Pipeline for Temporal Sentence Grounding in Compressed Videos
- Judging a video by its bitstream cover
- Shot Boundary Determination on MPEG Compressed Domain and Story Segmentation Experiments for TRECVID 2004
- CVT5: Using Compressed Video Encoder and UMT5 for Dense Video Captioning
- Online Misinformation Video Detection: A Survey
- Combating Online Misinformation Videos: Characterization, Detection, and Future Directions
- Mitigating Temporal Misalignment by Discarding Outdated Facts
- Mitigating Temporal-Drift: A Simple Approach to Keep NER Models Crisp
- TimeLMs: Diachronic Language Models from Twitter
- HealthFC: Verifying Health Claims with Evidence-Based Medical Fact-Checking
- Natural Supplements for H1N1 Influenza: Retrospective Observational Infodemiology Study of Information and Search Activity on the Internet