When a journal discovers that an author failed to disclose a consulting fee, a founding stake in a company, or a seat on the journal's editorial board, it publishes a correction. The original paper stays online, and a short notice appears separately. A language model trained on the scientific web reads both. It probably memorizes the long, repeated, heavily cited original much better than the short notice that appears once and is rarely linked. We propose to measure this gap directly and to test how it can be closed. The question is whether a model that can describe a paper's findings also knows the paper's corrected competing-interests statement, or whether it confidently repeats the stale "the authors declare no competing interests."
Our claim is that correction notices behave like the rare "minority" records that unlearning research has shown to be handled worst. This makes the stale-disclosure problem predictable from how often the correction appears relative to the original. It also means the obvious fix is the wrong one. Erasing the paper from the model ("unlearning") destroys valid findings that a disclosure correction does not invalidate. The right operation is narrower: replace one attribute of a memorized document and keep the rest. We build a controlled benchmark for this replace-don't-erase operation. We then compare retrieval, continued training, preference-based unlearning, and representation steering on it, and we treat retrieval as the default that parametric methods must beat.
Where this sits
The memorization literature supplies both the motivation and the methods. The Janus Interface shows that information a model appears to have forgotten survives in intermediate layers. In that work, fine-tuning GPT-3.5-Turbo on 10 examples recovered 69.9% of target Enron emails. This constrains our design: any "fixed" disclosure must be tested for whether the stale version can be brought back. Underestimated Privacy Risks for Minority Populations finds that rare records suffer at least 20% higher leakage under unlearning than average-case estimates suggest. This motivates our frequency hypothesis, since correction notices are exactly such rare records. Leak@k shows that greedy decoding hides content still extractable by sampling, so we score stale disclosures under sampling as well as greedy decoding. TOFU supplies the template of fictitious entities fine-tuned into a model with known ground truth, and documents knowledge entanglement: removing an author damages neighboring facts. Our benchmark copies the design but changes the target. TOFU asks the model to forget a whole profile; we ask it to update one field of a document while preserving the others.
On the publishing side, the file contains real instances of every correction type we model. These include an omitted company founding role in a systematic review of re-identification attacks, undisclosed consulting fees in the hyperpolarized pyruvate study, industry financial ties in the perovskite crystallization paper, and editorial-board overlap in the extended producer responsibility review and the 2D-material polarizer paper. The "window of risk" between publication and correction, discussed in the Drosophila haplotype correction, explains why filtering the corpus before training cannot catch these: the correction often does not exist yet at crawl time.
The closest existing work is external, not parametric. VERIRAG audits retrieved study summaries for methodological flaws at test time, and SciFact verifies claims against retrieved abstracts. Neither asks what the model's weights already believe about a paper, and neither handles financial or editorial conflicts. Content-based CoI detection on Wikipedia detects promotional text with classifiers. That task is different: it judges a text, whereas we track a document's corrected metadata inside a model. We found no work that measures whether models absorb post-publication corrections.
Research Program
The first component is a synthetic benchmark we call fictitious papers with corrections. We generate several hundred fictitious papers, each with a title, authors, a methods description, a few quantitative findings, and an original competing-interests statement. The statement types are modeled on the real corrections above: a founding stake, consulting fees, industry funding, and editorial-board membership. For a chosen subset we generate a correction notice that changes only the disclosure. We fine-tune open-weight models on a mixture in which each original paper appears with controlled repetition, mimicking preprints, mirrors, and citing summaries. Each correction appears at a controlled ratio to its original, from 1:1 down to 1:50. We also vary whether the notice repeats the paper's title or refers to it only by DOI. Because everything is fictitious, there is no contamination from pretraining, and the effect of frequency and linking can be estimated causally.
The second component is a real-world probe. We take papers that have Crossref-registered competing-interest corrections, seeded by the cases in the file. We then query open-weight models whose pretraining corpus snapshot dates are documented, separating papers whose correction predates the snapshot from those whose correction came after it. Only the first group could in principle have been learned. This arm checks whether the synthetic frequency curve predicts behavior in real pretraining.
The third component is the intervention comparison. It asks which method can make the model state the corrected disclosure, stop producing the stale one even under sampling, keep the paper's findings intact, and hold up under downstream fine-tuning.
Baselines
The naive reference point is doing nothing: we query the fine-tuned model as it is. This gives the stale-disclosure rate, meaning how often the model asserts the original "no competing interests" statement after having seen the correction. It also gives the finding-recall rate, against which all interventions are measured for collateral damage.
The strongest competitor for question answering is retrieval augmentation with an explicit correction lookup. When a paper is mentioned, its DOI's correction relations are fetched and the notice is placed in context. In the synthetic arm this lookup is a perfect oracle. In the real arm it uses whatever relation metadata exists. It is strong because it requires no weight change and is trivially correct when the paper is identified. Its known weakness is identification. SciFact reports verification falling from 60.6 F1 with oracle abstracts to 39.5 F1 with open retrieval. Even the more elaborate VERIRAG pipeline runs at about 25 seconds median per summary. We will run retrieval both with the paper's identity given and in an open setting. In the open setting the model writes an unprompted related-work paragraph, and retrieval must work out which papers it drew on.
The simplest parametric baseline is continued fine-tuning on the correction notices, upweighted. We include an unlearning baseline: NPO applied to the stale disclosure sentences, followed by fine-tuning on the correction. NPO is the method reported as the only one to keep TOFU forget quality at p > 0.05 on Forget10 while preserving utility. We also include RMU and its adaptive variant (On Effects of Steering Latent Representation for Large Language Model Unlearning), restricted to the disclosure spans. RMU's reported figure on WMDP-Bio for Zephyr-7B is a drop from 63.7% to 31.2% while MMLU moves from 58.1% to 57.1%. Adaptive RMU reaches 23.7%. These numbers come from a different task, and no reported result was found on Lacuna for any method on correcting memorized document metadata.
The ablation that isolates our key ingredient is whole-paper unlearning, meaning NPO applied to the entire original paper. This tests whether the narrow replace-one-field target matters. If whole-paper unlearning removes the stale disclosure but also removes the findings, entanglement is confirmed and the narrow target is necessary.
All methods use the same base models, data, and number of gradient steps. Hyperparameters are tuned on a held-out set of fictitious papers under a fixed search budget per method. A parametric method beats retrieval if it reaches comparable corrected-disclosure accuracy in the identified setting, a lower stale-disclosure rate under sampling in the open setting, and finding recall within a few points of the do-nothing model.
Why not just…?
Why not just use retrieval with a correction lookup? This is the real threat. If the paper is identified, retrieval will probably win, and we expect to report that. The open question is generation that does not name its sources: summaries and related-work paragraphs in which the stale disclosure comes out of the weights and nothing triggers a lookup. The open-setting comparison settles this. If retrieval with post-hoc source identification matches parametric methods there too, the conclusion is that disclosure correction belongs in retrieval infrastructure, and the measurement study stands on its own.
Why not just fine-tune on the correction notices? It may work at high correction-to-original ratios. The minority-leakage results here and the Janus recovery results here suggest the stale version will stay extractable under sampling and will return after unrelated downstream fine-tuning. Both effects are measured directly by our Leak@k-style scoring and a Janus-style refinetune test.
Why not just filter the corpus or retrain? The correction usually postdates the crawl, as the window-of-risk discussion in the Drosophila haplotype correction notes, so there is nothing to filter at training time. Retraining for each correction is not feasible. The real-world arm quantifies how many corrections postdate typical snapshots.
Why not just prompt the model to check for conflicts? A prompt cannot tell the model about a correction it never encoded, and refusal-style behavior is shallow according to the WMDP study. We include a disclosure-checking system prompt as an extra row in the do-nothing condition.
Decisive Checks
The central measurement is the stale-disclosure rate as a function of the correction-to-original ratio and of linking. The prediction, by analogy with minority-record leakage, is that the stale rate stays high until the correction approaches parity with the original, and is worse when the notice does not repeat the title. A flat curve would refute the minority-data framing.
The second check is a three-way trade-off for each intervention. It combines corrected-disclosure accuracy, stale-disclosure leakage under sampled decoding, and recall of the paper's findings, together with a Janus-style test: after fine-tuning on 10 to 30 unrelated paper–disclosure pairs, does the stale statement return?
The third check is transfer. Does the frequency curve fitted on fictitious papers predict which real corrected papers a pretrained model gets wrong? If it does, the curve becomes a tool for publishers: it estimates which corrections deployed models are least likely to have absorbed.
References
- The Janus Interface: How Fine-Tuning in Large Language Models Amplifies the Privacy Risks
- Underestimated Privacy Risks for Minority Populations in Large Language Model Unlearning
- Leak@$k$: Unlearning Does Not Make LLMs Forget Under Probabilistic Decoding
- TOFU: A Task of Fictitious Unlearning for LLMs
- Correction: A Systematic Review of Re-Identification Attacks on Health Data
- Author Correction: Hyperpolarized 13C pyruvate magnetic resonance spectroscopy for in vivo metabolic phenotyping of rat HCC
- Correction: Steering the crystallization of perovskites for high-performance solar cells in ambient air
- Correction to: Achievements and policy trends of extended producer responsibility for plastic packaging waste in Europe
- Publisher Correction: Advanced optical polarizers based on 2D materials
- Author Correction: Mitochondrial haplotypes affect metabolic phenotypes in the Drosophila Genetic Reference Panel
- VERIRAG: A Post-Retrieval Auditing of Scientific Study Summaries
- Fact or Fiction: Verifying Scientific Claims
- Content-Based Conflict of Interest Detection on Wikipedia
- Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearning
- The WMDP Benchmark: Measuring and Reducing Malicious Use with Unlearning
- On Effects of Steering Latent Representation for Large Language Model Unlearning