When a paper is corrected, the correction usually reaches only the publisher's landing page. Downloaded PDFs, aggregator indexes, preprint mirrors, and the large text corpora used to train and evaluate scientific language models keep the old version. Nothing tells the holder of a copy that their copy is stale. We propose a short digital "fingerprint" of the claim-bearing content of a paper, published alongside the version of record. Anyone holding a copy can recompute it locally and check it against the registry using a few hundred bytes. The fingerprint is built to change when the kind of edit errata actually make occurs: a swapped figure label, a changed number, a flipped "increased"/"impaired." It is built to stay stable under the noise that comes from re-parsing, reformatting, or a different PDF extractor.
The design comes from communication complexity, the study of how many bits separated parties must exchange to compute something about their combined data. A basic lesson there is that randomness buys large savings for checking whether two distant copies agree. Deterministic comparison essentially requires shipping the data. Shared random "fingerprints" need only a handful of bits. The scientific setting adds one complication that the theory has also studied: the two parties never hold identical inputs, because their copies and their extraction software differ slightly. Our claim is that a randomized check works on real scientific text only after the text is reduced to canonical claim atoms. Generic similarity sketches cannot distinguish a two-token correction from equal-sized parsing noise. Deterministic or fixed-hash checks can be gamed by anyone who wants a bad copy to pass.
Core Claim
Represent each document as a set of normalized claim atoms: numeric assertions bound to their variable and unit, directional relations between entities, and figure-label-to-axis bindings. Then audit a copy against the version of record by comparing randomized sketches of the two atom sets, using a fresh seed that the registry issues at audit time. This gives three things together. It detects correction-relevant edits at a low false-alarm rate. Its communication is proportional to the number of changed atoms, not to document length. It resists edits crafted to collide with a fixed hash. We expect that removing the canonicalization step, the randomness, or the seed freshness each breaks a different one of these properties, and the experiments are designed to show which.
Where the Idea Comes From and What It Does Not Claim
Two pieces of theory supply the shape of the method. In the three-party "number-on-forehead" model, Kelley, Lovett and Meka Explicit separations between randomized and deterministic Number-on-Forehead communication give an explicit function that a public-coin protocol solves with $O(1)$ bits by fingerprinting what each party sees, while every deterministic protocol needs $\Omega((\log N)^{1/3})$ bits. Pitassi, Shirley and Shraibman The Strength of Equality Oracles in Communication show how much an equality test adds to a protocol's power, and that a single equality call suffices without loss of generality. We do not claim that peer review or errata propagation has number-on-forehead structure. The research file's suggestion along those lines strikes us as an analogy, not a reduction. What we take from this work is the operational lesson: if a verification problem can be reduced to equality tests on small, canonical objects, randomized fingerprints make it cheap. Our engineering problem is to perform that reduction on messy scientific text.
The theory that most directly constrains the design is Ghazi and Sudan's The Power of Shared Randomness in Uncertain Communication. They study parties who hold slightly different versions $f \approx g$ of the function to be computed. In that setting, public shared randomness achieves $O(1)$ communication, private randomness needs $\Omega(\log^{(t)} n)$, and the usual trick of trading public coins for private ones fails. They also show that imperfectly shared randomness suffices, but only under product distributions. This maps literally onto our setting. The registry and the copy holder run different versions of the atom extractor, so they compute slightly different atom sets even from the same text. Ghazi and Sudan therefore predict that the shared seed is not a convenience. We test that prediction, and because real atom distributions are far from product-structured, we treat their positive result as a hypothesis, not a guarantee.
On the correction side, the problem comes from three sources. Shen Correction to: Copyright Volume 31, Issues 1–4, Acta Mechanica Solida Sinica documents that errata published in supplementary issues do not propagate back to downloaded PDFs or indexing services. Bevers et al. Author Correction: Mitochondrial haplotypes affect metabolic phenotypes in the Drosophila Genetic Reference Panel show the typical substantive erratum: two swapped haplotype labels on an axis, a tiny textual change with a large change in meaning. The field's own record is not immune: Errata for: "On randomized one-round communication complexity" is a passive notice that no downstream copy can detect. The Alexandria proposal Treat societally impactful scientific insights as open-source software artifacts would version papers like software. That solves staleness for copies living inside the repository, but not for the PDFs and derived corpora that have already left it, which is the case we target.
Research Program
The work has four parts.
Atom extractor. The first part builds the extractor. It finds numbers with their bound variable, unit, and comparison. It finds directional relations between entities, which is the "impaired" versus "increased" distinction that Fact or Fiction: Verifying Scientific Claims identifies as a recurring failure of claim verifiers. It also finds figure and table label bindings. Each atom is normalized to a canonical string: units converted, entity names lemmatized, relation polarity made explicit. Normalization is what keeps parser noise out of the atom set.
Sketch protocol. The registry publishes a seed $s$ at audit time. Both sides hash their atom sets under $s$ into a small reconciliation sketch, which encodes a set so that two sketches can be subtracted to recover the atoms where the sets differ. The size of the sketch grows with a target number of differences $d$, roughly $O(d \log U)$ bits for atoms drawn from a universe of size $U$. That growth follows from each recovered difference needing an identifier of about $\log U$ bits. A mismatch flags the copy as stale and returns the atoms that differ, which tells a reader what changed. Seeds are rotated per audit, so an adversary who knows yesterday's hash cannot construct an altered copy that passes today.
Cross-extractor analysis. We deliberately run mismatched extractor versions on the two sides. We measure how false alarms grow with extractor disagreement, and compare shared-seed protocols against protocols in which each side hashes with its own seed and must therefore send more.
Downstream audit. We apply the protocol to derived corpora such as the claim and abstract collections used for scientific claim verification. We estimate how many stored items lag a later correction. For flagged items, we use the citation-role labels of Öztürk et al. Assessing the Impact of Retracted Articles on the Reliability of Citing Literature: A Mixed-Based Proposal to tell whether citing papers depend on the changed atom as a core claim or merely mention it as background or critique.
Data and Evaluation Construction
No existing benchmark poses this exact task, so we assemble it from resources in the file.
Positives (real substantive edits). These come from AutoSupervision: Closing the Feedback Loop in Scientific Workflows with Grounded Revision Verification. Its 847 revision episodes and 8,790 concern instances pair a manuscript before and after revision, with grounded locations of the text that changed. Revisions are broader than errata, so we keep the concerns whose grounded change touches a number, a relation, or a figure binding.
Negatives (realistic noise). We re-parse the same version under the clean, degraded, and real-world conditions of PureDocBench, and with two different extractors.
Hard synthetic positives. We take SciFact claims and their rationale sentences and flip relation direction or perturb a single number. These test the smallest edits we care about.
Real-world errata. A small hand-curated set of published author corrections with recoverable before-and-after text, of the Bevers type, serves as a final check.
Metrics. The primary metric is true-positive rate at a 1% false-alarm rate. False alarms dominate cost when millions of copies are audited. We also report bits communicated per audit, accuracy at localizing the changed atom, and each of these broken out by PureDocBench noise tier.
Baselines
Naive reference: exact hashing. This is the do-nothing point that an integrity checker would reach for first: hash the extracted full text and compare. It uses 256 bits and has perfect recall, but every re-parse produces a mismatch. We report its false-alarm rate on the noise negatives. We expect it to be near total, which is why the problem is not already solved.
Registry lookup. Given a DOI, query a correction or retraction database such as the Retraction Watch database the file describes. We run it on the subset of copies that carry a DOI, under the same audit budget. On that subset it is an upper bound. Our method beats it only on items without a usable link, for example derived text or unlinked errata, and on the question of what changed, which a lookup cannot answer.
Generic near-duplicate sketches. The strongest randomized competitor is a MinHash or SimHash sketch over word shingles at matched size: 64, 256, and 1024 bytes. It is randomized and noise-tolerant, so it isolates whether claim-atom canonicalization is the ingredient that matters. We tune its shingle length and threshold on a held-out split with the same budget used for our thresholds. Beating it means a higher true-positive rate at 1% false alarms at every matched sketch size, with the largest gap on the PureDocBench degraded tier.
Strongest existing method: full-text LLM comparison. We give a frontier model both full texts and ask it to identify substantive changes. This is the deterministic, ship-everything end of the spectrum. The closest reported number is on AutoSupervision's grounding sub-task: top models score "above 0.70" at identifying the evidence for a revision, while base GPT-4o-mini scores 0.085, according to AutoSupervision. There is no reported result on Lacuna for detecting substantive change between noisy document copies. We will run this baseline ourselves on identical pairs. Beating it means matching its detection rate within a few points while sending at least two orders of magnitude fewer bits, and without requiring that either party obtain the other's full text.
Ablations. Two ablations isolate the key ingredients. The first keeps atoms but uses a fixed public hash, which tests what seed freshness buys under adversarial collision edits. The second keeps atoms and randomness but gives each side its own seed, which tests the Ghazi–Sudan prediction about shared randomness under extractor mismatch.
Why not just…?
Why not just look up the DOI in a correction registry? For a paper with a DOI and a properly linked notice, this is simpler and better, and we will say so in the results. It falls short in three documented ways. Shen shows that errata in supplementary issues often never link back. Cheng et al. Retractions relevant to COVID-19: does the retraction rate jump during the pandemic? find that over 30% of retraction notices give no reason, so a lookup says that something happened but not what. And text that has been extracted into corpora no longer carries reliable identifiers. The real threat is that the unlinked remainder could be small. We will measure its size on our curated errata sample before claiming broad impact, and if it is negligible, the contribution narrows to localizing what changed.
Why not just have a frontier LLM compare the two versions? It may detect changes well, but it requires shipping and reading both full texts for every audit, which is exactly the deterministic cost we are trying to avoid at corpus scale. AutoSupervision also reports that models are misled by author response letters that overstate changes and that judging sufficiency peaks around 0.501. Our task is narrower than sufficiency, however, so those numbers do not settle the question. The head-to-head comparison at matched pairs, with bits counted, will.
Why not just use standard near-duplicate detection like MinHash? This is the most serious alternative because it is already randomized and cheap. Our argument is that a two-label swap changes a tiny fraction of shingles, about as many as a degraded re-parse does, so no threshold separates the two. If MinHash at matched size performs as well as atom sketches on the AutoSupervision and SciFact-flip positives, then canonicalization adds nothing. In that case we would drop the extractor and study only seed freshness and localization.
Decisive Checks
The proposal succeeds if four things hold. First, at 256 bytes, atom sketches achieve a clearly higher true-positive rate at 1% false alarms than shingle MinHash on directional and numeric flips, and the gap holds on the PureDocBench real-world tier. Second, detection comes within a few points of full-text LLM comparison at a hundredth of the communication. Third, a fixed-hash variant is defeated by constructed collision edits that fresh seeds catch. Fourth, the own-seed variant shows growing cost or error as extractor disagreement increases, as the uncertain-communication theory predicts. If the first check fails, the core claim is wrong. If only the fourth fails, the method still stands, but the theoretical motivation for sharing seeds does not transfer to real text.
References
- Explicit Separations between Randomized and Deterministic Number-on-Forehead Communication
- The Strength of Equality Oracles in Communication
- The Power of Shared Randomness in Uncertain Communication
- Correction to: Copyright Volume 31, Issues 1–4, Acta Mechanica Solida Sinica
- Author Correction: Mitochondrial haplotypes affect metabolic phenotypes in the Drosophila Genetic Reference Panel
- Errata for: "On randomized one-round communication complexity"
- Treat societally impactful scientific insights as open-source software artifacts
- Fact or Fiction: Verifying Scientific Claims
- Assessing the Impact of Retracted Articles on the Reliability of Citing Literature: A Mixed-Based Proposal
- AutoSupervision: Closing the Feedback Loop in Scientific Workflows with Grounded Revision Verification
- How Far Is Document Parsing from Solved? PureDocBench: A Source-TraceableBenchmark across Clean, Degraded, and Real-World Settings
- Retractions relevant to COVID-19: does the retraction rate jump during the pandemic?