Does Benchmark Contamination Fade Faster Than Our Ability to See It?

When a test set leaks into a language model's training data, the model's score on that test goes up. Almost every contamination audit, though, inspects a single finished checkpoint. Real training runs are long and sequential: a benchmark that was crawled into the early pre-training data is followed by hundreds of billions of clean tokens, then by domain-specific continued training, then by fine-tuning. This proposal asks a simple question about that sequence. As clean training accumulates after a leak, two things decay. One is the score inflation the leak causes. The other is the statistical trace that contamination detectors look for, typically unusually low loss on the leaked items. Nobody has measured whether these two decay at the same rate. If the trace fades faster than the inflation, then a model can benefit from contamination that no current detector can find.

We propose a controlled study of this question. We inject benchmark items at a known point in training, continue training on clean data for measured token budgets and under different kinds of distribution shift, and track inflation and detectability side by side. The output is a pair of decay curves per contamination type, model scale, and continuation regime. From these we identify a regime we call "invisible inflation," in which accuracy on leaked items is still measurably higher than on matched unleaked items while detector AUROC (the probability that a detector ranks a leaked item above an unleaked one) has fallen near chance. The study treats the two research directions asymmetrically. Pre-training leakage, the worry that "novel" evaluation concepts were already seen during earlier training, provides the temporal lens. Benchmark contamination provides the measurements, the detectors, and the stakes.

Core Claim

Our hypothesis is that loss-based contamination signals decay with clean continued training faster than the accuracy benefit does. We expect the gap to be largest for ground-truth contamination, meaning leaked question, prompt, and answer together, and for larger models. If the hypothesis holds, single-checkpoint detection systematically under-reports contamination in models that saw benchmarks early. If it fails, meaning both quantities decay together, that is also useful: it would justify the common informal belief that early leaks wash out, and it would show that current detectors remain valid across long training runs.

Let $\Delta(t)$ be the accuracy on leaked items minus the accuracy on matched held-out items from the same benchmark, after $t$ clean tokens of continued training. Let $A(t)$ be the AUROC of a given detector at the same point. Write $t_\Delta$ and $t_A$ for the token budgets at which $\Delta$ falls to half its initial value and $A - 0.5$ falls to half its initial value. The quantity of interest is the ratio

$$\rho = t_\Delta / t_A,$$

computed per detector and setting. A value of $\rho > 1$ means inflation outlives detectability, and the width of the window between $t_A$ and $t_\Delta$ is the invisible-inflation regime.

Why This Gap Is Real

Three findings motivate the question. Investigating Data Contamination for Pre-training Language Models pre-trains GPT-2 models from scratch on a 3.3B-token subset of The Pile with injected benchmark data. It shows that ground-truth contamination inflates scores far more than text-only contamination, that inflation follows a U-shaped curve in the number of repetitions, and that 8- and 13-gram decontamination filters fail. It studies only a single training batch, with nothing after the leak. On The Fragility of Benchmark Contamination Detection in Reasoning Models shows that later training can erase detector signals while preserving the score gain. After one epoch of clean reinforcement learning with GRPO (a policy-gradient method with clipped importance ratios), LiRA AUROC falls by about 14 points from 89.1%, and reference-free detectors approach 50%. That paper attributes the erasure to the clipping mechanism specific to reinforcement learning. Whether ordinary next-token training on clean data, which has no clipping, produces a similar decoupling is untested, and that is exactly the training that follows most real leaks. Finally, Investigating Continual Pretraining in Large Language Models finds on the M2D2 corpus that small models are "spongy" and forget quickly, while Llama2-7B is "stubborn." This suggests that persistence of contamination should depend strongly on scale, which a single-scale study cannot capture.

The vision continual-learning literature shows the same structural problem from the other side. Revisiting Class-Incremental Learning with Pre-Trained Models shows that a frozen pre-trained model with class-mean prototypes (SimpleCIL) beats elaborate continual learners, because the "new" classes were already seen in pre-training. DESIRE reports results on ImageNet380-20T, a split built to exclude pre-training classes. Both treat earlier exposure as something to filter out. We instead measure how its effects and its traces persist.

The closest bridging work is CLDyB. It builds adversarial continual-learning task sequences, using Monte Carlo tree search over task orderings to maximize forgetting and minimize learning accuracy, and draws on photographic and Stable Diffusion XL-generated image pools to escape contamination. CLDyB explicitly provides no measure of how much contamination a backbone carries, and it is confined to vision classification. Our study does not construct harder benchmarks. It quantifies how the effect and the detectability of known contamination evolve over sequential language-model training. DCR is also relevant. It scores contamination at four levels (semantic, information, data, label), combines them into a factor between 0 and 1, and corrects accuracy to within 4% of clean baselines on SST-2, LIAR2, and GSM8K. Like the other detectors, it has only been validated at a static checkpoint. In our study it becomes one of the detectors whose decay we track.

Research Program

Base models and injection. Following Jiang et al., we pre-train GPT-2 small (124M) and GPT-2 large (774M) on a 3.3B-token Pile subset. For each benchmark, we inject a random half of the evaluation items and hold out the other half as matched non-members. We test two contamination types, text-only and ground-truth, at repetition factors of 1, 5, and 10. We leave out 20 repetitions because Jiang et al. already show degradation there.

Clean continuation. From each contaminated checkpoint, we continue training with no further leakage under three regimes. The first is in-distribution: more Pile data drawn from outside the 3.3B subset. The second is domain-shifted: sequential M2D2 domains. The third is shuffled M2D2, chosen because Yıldız et al. find that random ordering changes retention. We save checkpoints at geometrically spaced clean-token budgets up to several times the original pre-training budget, and evaluate inflation and every detector at each one.

Scale check. Training a 7B model from scratch is out of reach. At 7B we therefore use the continual-pretraining setting itself as the injection vehicle. We continue training Llama2-7B on M2D2 data with benchmark items mixed into one early domain, then continue on clean domains. Yıldız et al. report that Llama2-7B needs roughly 100MB of domain text before it measurably adapts, so we size the injected domain accordingly.

Benchmarks. We use the static benchmarks from Jiang et al. (SST-2, MMLU, CNN/Daily Mail, SQuAD V1) so that our step-zero numbers can be checked against theirs. We add GSM8K as the benchmark DCR uses. We also add DyVal tasks (DyVal), which generate reasoning problems from directed acyclic graphs at controlled depth. DyVal plays a special role. Because its generator produces unlimited fresh problems of matched difficulty, it gives a held-out set that is guaranteed clean at exactly the complexity of the leaked items. This removes the worry that our member and non-member halves differ in difficulty. The PromptBench library supplies the DyVal generators and the Meta Probing Agent perturbations.

Baselines

The naive reference point is an uncontaminated twin model trained on the identical clean token stream. Its member-versus-held-out gap should be about zero at every checkpoint, which fixes the noise floor for $\Delta(t)$. It also defines the "do nothing" policy: assume early leaks have washed out and report raw scores. Our measurements test that assumption directly.

The strongest existing detector is LiRA, a likelihood-ratio membership attack that compares the target model's loss on an item against reference models trained without it. Fragility reports 89.1% AUROC for LiRA on freshly fine-tuned contamination. We expect it to stay the strongest loss-based detector here. We will run it with reference models trained on the same clean streams at the same checkpoints, which gives it an advantage a real auditor would not have. We also include perplexity loss and Min-K%, which scores an item by the average log-probability of its least likely tokens. These represent the reference-free detectors Fragility found most fragile. All loss-based detectors get the same evaluation items and the same compute. Hyperparameters, such as the K in Min-K% and the number of LiRA reference models, are tuned once at the step-zero checkpoint and then held fixed. That way, a falling AUROC reflects decay of the signal and not retuning.

As the strongest corrected-accuracy method, we run DCR's four-level probes. For DCR we track two things over time: the factor itself, and how far the adjusted accuracy lies from the uncontaminated twin. DCR's reported static fidelity, within 4% of clean baselines, is the standard to beat. If DCR stays within 4% across all continuation budgets, it is robust to the decay we study.

As a behavioral, loss-free detector, we use the Meta Probing Agent perturbation gap from Dynamic Evaluation of Large Language Models by Meta Probing Agents. That method paraphrases questions, permutes options, and adds distractors, and the paper reports a 15.7% drop for GPT-4-Turbo on MMLU under full probing. Here, the detector score is the accuracy drop on an item's perturbed variants, with the semantic-equivalence check carried out by the same judge for every item. For DyVal items, the analogous score is the gap between an original problem and freshly generated problems of the same depth.

The main ablation isolates the ingredient that is new in this study, the clean continuation. It compares the in-distribution and domain-shifted regimes at equal token budgets. If the decay ratio $\rho$ depends on shift rather than on token count, then the distance traveled in data space, and not the time elapsed, is what removes contamination. A second ablation compares text-only and ground-truth injection. A detector "beats" the baseline if it keeps AUROC at least 10 points above chance at every checkpoint where $\Delta(t)$ is still statistically significant.

Why not just…?

Why not just assume that early leaks are forgotten? This is the working assumption of most decontamination reporting, and it is exactly what the experiment tests. The stubbornness of Llama2-7B under continued training reported by Yıldız et al. gives a concrete reason to doubt it at scale. The uncontaminated-twin comparison settles it: if $\Delta(t)$ reaches zero within modest clean budgets for every setting, the assumption holds and we will say so.

Why not just switch to dynamic benchmarks and stop detecting? This is a serious alternative. The survey Benchmarking Large Language Models Under Data Contamination shows HumanEval pass@1 rising from 0.19 to 0.82 under contamination while DyCodeEval stays flat at about 0.14. But the same survey documents a "complexity trap": many generators produce unstable difficulty. Procedural generation also covers formal tasks well and open-ended language understanding poorly, and thousands of existing static results still need auditing. Our study does not compete with dynamic benchmarks. It measures how long static scores stay untrustworthy, and it includes the dynamic-gap detector as a contender. If that detector tracks $\Delta(t)$ perfectly, the practical recommendation is to use dynamic twins wherever a generator exists, and we will report that.

Why not just apply DCR at the final checkpoint? DCR may well remain accurate after continued training. Its label-level probes test whether the model reproduces answers, which is closer to inflation than to loss. That outcome would be a positive finding about DCR. However, its hand-written probe templates have only been validated on static checkpoints of SST-2, LIAR2, and GSM8K. Tracking the fidelity of its adjusted accuracy across checkpoints answers this directly, without assuming it.

Why not just tighten n-gram filtering of the corpus? Jiang et al. show that 8- and 13-gram filters leave contaminated performance intact even in single-stage training. Filtering also does nothing for the question of models that have already been trained, which is the auditing case we target.

Decisive Checks

The first check is replication at step zero. Before any continuation, our contaminated GPT-2 large must reproduce the direction and approximate size of Jiang et al.'s MMLU result (24.3% with ground-truth contamination at 10 repetitions versus 22.8% clean) and their CNN/Daily Mail result (0.6996 versus 0.5968). Without this, later decay curves cannot be interpreted.

The central check is whether $\rho$ exceeds 1 with bootstrap confidence intervals that exclude 1 for LiRA and Min-K% under ground-truth contamination, and whether $\rho$ grows from 124M to 774M to the 7B continual setting. Our prediction fails cleanly if $\rho \le 1$ everywhere. A third check asks whether behavioral detectors (the perturbation gap and the DyVal fresh-twin gap) keep $\rho$ near 1 where loss-based detectors do not. That would mean that measuring what the model can do, rather than how surprised it is, is the robust signal over long training. A final check compares the invisible-inflation window under domain-shifted and in-distribution continuation at equal token counts. This tells practitioners whether later domain-specific training, which is common in deployment, makes earlier leaks harder to audit.