Forgotten but Not Gone: Relearning Probes for Benchmark Contamination Hidden by Post-Training

When a language model has seen benchmark test questions during training, its scores on that benchmark are inflated. Most tools for catching this "contamination" look for a memory trace, such as unusually low loss on the leaked questions. Recent work shows these traces can disappear. A later round of training on clean data, such as reinforcement learning on math, pushes standard detectors toward chance, while the inflated scores largely survive. We propose to treat this disappearance as a case of catastrophic forgetting: the loss of earlier learned behavior when a network is trained on something new. The continual-learning literature has two findings that bear on it. How much gets forgotten depends on how related the new task is to the old one. And forgotten knowledge is often only suppressed, so it comes back unusually fast when the model is retrained.

Our core claim is that contamination which surface detectors lose after post-training is still stored in the weights and can be measured by how quickly it comes back. We will build a controlled testbed that injects known contamination into open models. We will then post-train those models on tasks at deliberately varied semantic distance from the contaminated benchmark, and map when the surface signal and the score inflation come apart. The detector we test is a "relearning probe." We briefly fine-tune on half of the suspect benchmark items and measure how much loss drops on the other, untouched half, relative to fresh items from the same task. If the untouched half recovers faster than general task learning can explain, the model is holding a latent memory of that benchmark.

Where This Sits in Prior Work

The motivating evidence comes from On The Fragility of Benchmark Contamination Detection in Reasoning Models. That paper contaminates models through supervised fine-tuning and then applies clean reinforcement learning with GRPO. One epoch cuts LiRA detection by about 14% AUROC and loss-based detection by about 16%, starting from 89.1% for LiRA. It also shows that contaminating a distilled reasoning model with chain-of-thought data gives an 11.7% pass@1 boost while ten detectors average 50–58% AUROC. The paper attributes the cloaking to importance sampling and clipping. It does not ask whether the traces are erased or merely suppressed, and it does not vary how related the clean post-training task is to the contaminated one. Those are the two things we change.

Three continual-learning results explain why task relationship should matter here.

Together these predict that post-training close to the contaminated benchmark should erase surface traces fastest. None of them concerns contamination.

Continual Memorization of Factoids in Large Language Models supplies both a warning and a method. It shows that fact-on-fact interference is extreme: TriviaQA accuracy drops from 100% to 4.3% after training on LAMA. It also uses Logit Lens probing, which reads predictions out of intermediate layers, to show where memorized facts are stored. We reuse that probing to ask whether post-training removes contamination from the output layers while leaving earlier layers intact.

On the detection side, the probe is measured against three families of methods.

  • Context perturbation. Detecting Data Contamination in LLMs via In-Context Learning (CoDeC) reports a 99.9% dataset-level AUC by exploiting the fact that in-context examples lower confidence on memorized data. It was evaluated on static checkpoints, not after cloaking post-training.
  • Performance gaps. ConStat: Performance-Based Contamination Detection in Large Language Models measures the score gap on rephrased benchmark variants, reporting δ ≈ 8.3% for Mistral-7B on GSM8K. It depends on clean reference models.
  • Audit power. When Is Benchmark Contamination Detectable? Information Limits and Power-Calibrated Audits shows detectability scales as $\alpha\rho\sqrt{m}$. Here $\alpha$ is the fraction of the benchmark that was contaminated, $\rho$ is how behaviorally distinguishable seen items are from unseen ones, and $m$ is the number of audited items.

Read through that bound, our proposal is an attempt to recover a large $\rho$ after post-training has driven it toward zero.

Core Claim

We make two claims.

The first concerns task relationship. The rate at which surface contamination signals decay during post-training is controlled by the semantic distance between the post-training task and the contaminated benchmark. The rate at which score inflation decays is controlled differently. This opens a "cloaking zone" of related-but-not-identical post-training, in which detectors fail but scores stay inflated.

The second concerns detection. Inside that zone, contaminated items retain a relearning advantage that transfers across items of the same benchmark. This advantage separates contaminated from clean benchmarks even when loss-based and context-based detectors cannot.

The probe statistic is defined as follows. Split the suspect items into halves $A$ and $B$, and take a matched clean set $C$ from the same task. Fine-tune for $k$ small steps on $A$ only. Then compute

$$ R = \frac{1}{|B|}\sum_{x\in B}\big[\ell_0(x)-\ell_k(x)\big] \;-\; \frac{1}{|C|}\sum_{x\in C}\big[\ell_0(x)-\ell_k(x)\big], $$

where $\ell_0$ and $\ell_k$ are the model's loss on an item before and after the $k$ steps. The first term is how much the untouched suspect half recovers. The second term subtracts the recovery that ordinary task learning produces on items the model has never seen. A positive $R$ means the untouched suspect half recovers more than the task in general, which is evidence of shared latent memory for that benchmark.

The matched clean set $C$ is built differently depending on the setting. In the controlled testbed it is held-out original items we know were never injected. In real audits it is ConStat-style rephrasings or post-cutoff items.

Research Program

The testbed starts from open models: Qwen2.5-1.5B-Instruct, a Llama-family 8B model, and DeepSeek-R1-Distill-Llama-8B, the last to match the fragility study. We inject contamination by supervised fine-tuning on a known half of the GSM8K or MMLU test items, holding out the other half as a clean control. We vary both the fraction injected and whether the leaked text is verbatim answers or chain-of-thought solutions.

Each contaminated checkpoint is then post-trained on clean data. The data is chosen from a graded ladder of distances from the contaminated benchmark:

Distance Clean post-training data (for GSM8K contamination)
Same distribution Clean GSM8K training split
Near MathQA
Middle ARC-Challenge
Far SST-2 and AG News classification
Unstructured FineWeb text

We run this ladder under both supervised fine-tuning and GRPO. Distance is measured, not just assigned. For each post-training set we record the embedding distance to the benchmark and the cosine similarity of its gradients with the gradient on benchmark items, so the results can be expressed as a curve over a continuous quantity.

At checkpoints throughout post-training we track three curves:

  • Score inflation: accuracy on contaminated items minus accuracy on the held-out clean items.
  • Surface detectability: AUROC for loss, Min-K%, LiRA, and CoDeC.
  • Latent detectability: AUROC of the relearning statistic $R$ across many contaminated and clean benchmark subsets.

We also run Logit Lens readouts per layer to see whether post-training removes contamination from the upper layers first.

Finally, we test a confound specific to CoDeC. CoDeC's in-context examples come from the target dataset. The in-context forgetting result of Understanding Generalization and Forgetting in In-Context Continual Learning suggests that confidence drops can also arise from interference between similar tasks, independent of memorization. We therefore rerun CoDeC with context drawn from near and far tasks on clean benchmarks to measure its false-positive rate.

Baselines

The naive reference point is loss thresholding and Min-K% probability applied to the final post-trained model, with no accounting for post-training. This is what an auditor typically runs today. We treat it as the floor.

LiRA is the strongest gray-box memorization detector in the fragility study. We give it shadow models trained with the same total compute that the relearning probe spends on its fine-tuning steps. The reported numbers from On The Fragility of Benchmark Contamination Detection in Reasoning Models are:

  • 89.1% AUROC before post-training;
  • roughly 14 points lower after one epoch of GRPO;
  • 50–58% average AUROC across ten detectors on chain-of-thought-contaminated reasoning models.

We will reproduce the GRPO-cloaked setting and require the relearning probe to reach at least 75% AUROC there. That is well above the reported collapse and near LiRA's uncloaked level. The probe must also hold a false-positive rate at or below LiRA's on the clean controls.

CoDeC is the strongest reported detector overall, with 99.9% dataset-level AUC in Detecting Data Contamination in LLMs via In-Context Learning. No reported result was found on Lacuna for CoDeC after cloaking post-training. We run it with its published context sizes on every checkpoint. To count as beating CoDeC, the probe must have higher dataset-level AUC inside the cloaking zone and a lower false-positive rate under near-task context.

ConStat is the strongest performance-based competitor, with δ ≈ 8.3% on GSM8K for Mistral-7B in ConStat: Performance-Based Contamination Detection in Large Language Models. We run it using the same rephrased items that form our clean set $C$ and the same reference models across all conditions.

Two ablations isolate the key ingredient.

  • Task-only fine-tuning. We fine-tune on clean items of the same task instead of on the suspect half $A$. If $R$ stays positive, the probe is measuring task learning, not contamination.
  • Direct versus cross-item recovery. We compare recovery on $B$, the untouched half, with recovery on $A$, the trained half, paired with rephrase controls. This tells us whether latent memory is shared across items or local to each item.

All probe hyperparameters, meaning step count, learning rate, and split size, are tuned on a separate calibration benchmark and then frozen.

Why not just…?

Why not just run CoDeC, which already reaches 99.9% AUC? Its reported result is on static checkpoints. Its mechanism, the perturbation of narrow memorization minima, is exactly what post-training reshapes. Its signal may also partly reflect in-context interference between similar tasks rather than memory. The ladder experiment measures CoDeC's AUC inside the cloaking zone directly, and the near-context control on clean benchmarks measures its false positives. If CoDeC stays above 95% throughout, the detection half of this proposal is unnecessary. We would say so and report only the forgetting characterization.

Why not just use ConStat, since what we care about is inflated scores, not memory? This is a real threat. ConStat measures exactly the inflation that persists after cloaking. Its weaknesses are that it needs uncontaminated reference models, and that contamination which generalizes to rephrasings, as chain-of-thought contamination may, would shrink the gap it relies on. We run ConStat in every condition. If it detects the cloaked models as well as the relearning probe does, we conclude that performance-based auditing suffices, and the contribution becomes the map of when surface detectors fail.

Why not just audit more items so weak loss signals become significant? The power bound in When Is Benchmark Contamination Detectable? Information Limits and Power-Calibrated Audits scales with $\rho\sqrt{m}$. When $\rho$ collapses toward zero, as the 50–58% AUROC figures indicate, the required $m$ exceeds the size of real benchmarks. We will use that paper's power planner to report, for each detector, the number of items needed for 0.8 power at each checkpoint. That makes the comparison with the probe quantitative rather than rhetorical.

Why not just search the training data with n-gram overlap? That requires access to the training corpus, which audits of released models usually lack. Task Contamination: Language Models May Not Be Few-Shot Anymore and ConStat both show that paraphrased and task-level exposure escapes string matching.

Decisive Checks

Three outcomes would settle the proposal.

The first is the forgetting map. If surface-detector AUROC decays fastest for near-distance post-training while score inflation decays more slowly, the cloaking-zone claim holds. If both decay together at every distance, cloaking is simply forgetting, and inflation is not a lasting concern once traces are gone. We would report that as a negative result.

The second is the probe. It succeeds if $R$ separates contaminated from clean benchmarks at 75% or higher AUROC in GRPO-cloaked and chain-of-thought-contaminated models, while the task-only fine-tuning ablation stays near zero. If cross-item recovery is absent but direct recovery is present, we fall back to the item-local variant and report the reduced scope.

The third is the layer readout. If Logit Lens shows contamination vanishing from late layers while remaining readable in earlier ones after near-task post-training, that would give the latent-memory account a mechanism, in line with the layer-storage findings of Continual Memorization of Factoids in Large Language Models.