Can Scientific Claims Be Removed from Large Language Models? A Systematic Evaluation of Claim-Level Unlearning

Snigdha Paul, Manasi Patwardhan, Arman CohanarXiv 2026 · arXiv

TL;DR

Large Language Models (LLMs) frequently internalize scientific claims that are later retracted or disproven, risking the spread of misinformation. This paper introduces SciUnlearn, a novel benchmark demonstrating that current machine unlearning methods only superficially suppress specific wordings of a claim rather than erasing the underlying scientific concept.

What Problem Does This Paper Address?

LLMs are trained on static snapshots of scientific literature, but science is inherently dynamic and self-correcting. When a breakthrough is later debunked or a foundational paper is retracted due to error or fraud, the outdated knowledge remains baked into the model's weights.

While "machine unlearning" has been studied for removing private data or copyrighted material, applying it to scientific claims introduces a unique challenge: structured conceptual knowledge. A scientific claim isn't just a single string of text; it can be paraphrased, reframed, and queried in countless ways. Current unlearning methods typically focus on instance-level forgetting (forgetting a specific sentence or document). This paper investigates whether LLMs can actually unlearn an abstract claim-level concept, or if they are simply learning to ignore specific phrasings.

Methods

To rigorously evaluate claim-level unlearning, the authors design a clever experimental setup. For any given scientific claim, they generate multiple paraphrased question-answer (QA) pairs and split them into two disjoint sets: Forget Set 1 ($F_1$) and Forget Set 2 ($F_2$).

An unlearning algorithm is applied using only the questions in $F_1$. If the algorithm successfully removes the underlying scientific knowledge, the model should lose its ability to answer questions in both $F_1$ and $F_2$. If it only forgets $F_1$ while retaining high accuracy on $F_2$, it proves the model has only memorized a surface-level suppression pattern.

The authors benchmarked several representative unlearning algorithms:

  • Gradient Difference (GD): Performs gradient ascent on the forget set while maintaining gradient descent on a retain set.
  • Negative Preference Optimization (NPO) & SimNPO: Preference-based optimization techniques that penalize the model for generating target forget strings.
  • Retain-Aware Variants (NPO+RT / SimNPO+RT): Algorithms augmented with an explicit objective to preserve non-target knowledge.

These methods were evaluated via parameter-efficient tuning (LoRA) and full-parameter fine-tuning on OLMo-3-7B-Instruct and Llama-3-8B-Instruct.

Datasets

The authors introduced SciUnlearn, a dedicated benchmark built from the Dolma corpus (the pretraining data for OLMo) to ensure the models had actually encountered the claims during training.

Figure 1: SciUnlearn Dataset generation pipeline.

SciUnlearn consists of three domain-specific subsets:

  1. $\mathcal{D}_{cs}$ (Computer Science): 170 claims from 76 recent papers.
  2. $\mathcal{D}_{med}$ (Medical Science): 127 claims from 78 papers.
  3. $\mathcal{D}_{retracted}$ (Retracted Papers): 141 claims from 91 real-world papers retracted after 2024 (identified via the Retraction Watch database).

For each claim, an LLM pipeline generates 8 QA pairs spanning four distinct formats: Multiple Choice (MCQ), True/False (T/F), Fill-in-the-blank (FB), and Assertion-Reason (AR). To ensure the unlearning doesn't cause catastrophic forgetting of adjacent knowledge, the dataset includes internal retain sets ($R_{int}$, contextual text from the same paper) and external retain sets ($R_{ext}$, highly similar cited papers).

Figure 2: Example claim and QA in SciUnlearn dataset.

Key Insights and Ideas

  • The Illusion of Forgetting: Current unlearning methods act as a "surface-level patch." They successfully suppress the exact training instances used during the unlearning process but fail to generalize to semantically identical questions phrased differently.
  • The "Rootedness" of Citations: Highly cited papers are significantly harder to unlearn. Because their claims permeate the pretraining corpus through thousands of derivative, citing works, the knowledge is deeply embedded and redundant. Conversely, low-citation papers are far more fragile and susceptible to "cross-set spillover" (where unlearning $F_1$ accidentally triggers the forgetting of $F_2$).
  • Format Matters: Question formats requiring generation or logical linking (like Fill-in-the-blank or Assertion-Reason) show higher error rates after unlearning, whereas recognition-based formats (like MCQs) are highly resistant to forgetting.

Key Results

The quantitative results highlight a stark gap between instance-level and claim-level forgetting.

As shown in the full-parameter unlearning results on the medical dataset ($\mathcal{D}_{med}$):

Table 4: Results of LoRA unlearning using F1 and F2 for GD and NPO+RT on Dcs. ↓indicates lower is better (forgetting), while ↑indicates higher is better (retention and generalization). OLMo-3-7B-Instruct is referred as OLMO and LLAMA3-8B-Instruct is referred as LLAMA. Additional method results are provided in App. L.
  • Failure to Generalize: When OLMo was unlearned on $F_1$ using Gradient Difference (GD), its exact match (EM) score on $F_1$ plummeted from 78.59% to 35.78%. However, its performance on the unseen paraphrased set ($F_2$) barely budged, dropping only from 65.96% to 61.05%.
  • Retain Set Stability: Methods utilizing explicit retain objectives (like NPO+RT) successfully protected adjacent scientific knowledge ($R_{ext}$ and $R_{int}$) and maintained stable performance on general capabilities benchmarks like MMLU and ARC-Challenge.

Limitations and Future Work

The authors acknowledge three primary limitations:

  1. Optimization-Only Focus: The study only evaluates optimization-based fine-tuning methods. It does not explore mechanistic or representation-level unlearning approaches (such as activation steering or sparse autoencoders) which might better target internal claim representations.
  2. Sample Size: While valuable, the real-world retracted paper subset ($\mathcal{D}_{retracted}$) remains relatively small.
  3. Model Scale: Due to compute constraints, experiments were limited to 7B-8B parameter models, leaving the behavior of frontier-scale models an open question for future research.

Summary updated

Abstract

Language models (LMs) are trained on static scientific corpora, whereas scientific knowledge continuously evolves through correction and revision. Scientific claims encoded within these models may later become retracted, disproven, or updated by subsequent research, creating the risk of disseminating outdated information in scientific workflows. This creates a need for LMs to forget obsolete scientific claims. Machine unlearning offers a promising solution by enabling knowledge removal while maintaining overall model utility. Existing studies primarily investigate instance-level forgetting; however, scientific claims introduce additional challenges because they are interconnected, and continually evolving. To address this gap, we introduce the task of Scientific Claim Unlearning and present a new benchmark, SciUnlearn. We show that current unlearning approaches are unable to effectively eliminate claim-level knowledge and often achieve only superficial suppression, highlighting the need for specialized methods designed for structured knowledge removal.
BibTeX
@misc{paul2026can,
  author = {Snigdha Paul and Manasi Patwardhan and Arman Cohan},
  title = {{Can Scientific Claims Be Removed from Large Language Models? A Systematic Evaluation of Claim-Level Unlearning}},
  year = {2026},
  eprint = {2608.20960},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2608.20960},
}
Figure