Relearning Speed as a Contamination Signal That Survives Reinforcement Learning

When a language model has seen benchmark questions during training, its scores stop measuring reasoning and start measuring recall. The standard tests for this, which ask whether the model finds a test item unusually likely, stop working once the model goes through reinforcement learning (RL) post-training. RL reshapes the output probabilities so that seen and unseen items look alike. It does this while leaving the score inflation in place. We propose a different test, borrowed from meta-learning (training a model so that a few gradient steps on a small "support set" of examples adapt it to a new task). The test asks how fast the model relearns. We take a handful of items from the suspected benchmark and fine-tune the model on them for a few low-rank gradient steps. We then measure how much the model's loss drops on other, held-out items from the same benchmark. We compare that drop with the drop on matched items the model cannot have seen, and with the drop in an uncontaminated reference model given the same update.

The intuition comes from memorization in meta-learning. A model that was trained on a task sits at a point in weight space from which that task is reachable in very few steps. RL post-training can suppress the surface evidence of memorization without moving the weights far from that point. If this is right, relearning speed ("savings") should expose contamination that likelihood-based detectors miss. The key claim is testable and falsifiable in one sentence: after one epoch of clean-data RL, item-level savings should still separate contaminated from clean items, while likelihood detectors fall to chance.

Why this gap is real

The contamination literature has converged on a specific failure. On The Fragility of Benchmark Contamination Detection in Reasoning Models contaminates Qwen2.5-7B-Instruct through supervised fine-tuning on competition math and science benchmarks, inflating pass@1 by 8.8%. It shows that the likelihood-ratio attack LiRA, a detector that calibrates an item's loss against reference models, catches this at 89.1% AUROC. After one epoch of GRPO, a policy-gradient RL method, run on clean data, LiRA drops by about 14 points and reference-free methods approach 50%, yet the inflation survives. The same paper reports that when a distilled reasoning model is contaminated with chain-of-thought data, it gains 11.7%, and all ten detectors it tests score 50 to 58% AUROC.

Detecting Data Contamination from Reinforcement Learning Post-training for Large Language Models finds the same collapse of perplexity-style detectors (AUC 0.42 to 0.53) on its RL-MIA benchmark. Its response is a behavioral probe, Self-Critique. It asks the model for an alternative reasoning path and compares the token-level entropy profiles of the two responses, reaching 0.70 AUC on Qwen2.5-7B-Instruct.

These two papers motivate the proposal. Both show that the information behind the inflation is still in the weights, but that no forward-pass statistic reads it reliably. Our probe reads the weights through their response to an update rather than through a single forward pass.

The meta-learning side supplies the mechanism and the framing. Meta-Learning Requires Meta-Augmentation shows that when training tasks can all be solved by one fixed function, the model stores the answers and ignores its support set. MAML accuracy on Mini-ImageNet falls to 30.2% without label shuffling, compared with 46.0% with it. A Representation Learning Perspective on the Importance of Train-Validation Splitting in Meta-Learning argues that high-capacity networks trained on too few tasks spend capacity storing task identities.

Benchmark contamination is the language-model version of the same thing: a benchmark that was part of training behaves like a meta-training task. Two further papers supply the practical tools. Meta-Adapters shows that tiny adapter modules on a frozen backbone are enough to express few-shot adaptation, tuning 0.6% of parameters. In-Loop Meta-Learning with Gradient-Alignment Reward shows that the inner product between one example's gradient and a look-ahead batch gradient separates corrupted from clean data in CIFAR training. Our savings statistic is a finite-step version of that alignment: an update on support items that helps the query item's loss is aligned with it.

Core Claim

Consider a candidate item $x$, an original model $\theta$, and a model $\theta_S$ obtained by $k$ low-rank gradient steps on a support set $S$ of other items from the same benchmark. We define the savings of $x$ as

$$s(x) = \big[L(x;\theta) - L(x;\theta_S)\big] - \big[L(x;\phi) - L(x;\phi_S)\big],$$

where $L$ is the loss on the reference solution tokens, and $\phi$ and $\phi_S$ are an uncontaminated reference model before and after the identical update. Subtracting the reference term removes the ordinary transfer that any model gets from practicing similar problems, in the same spirit as LiRA's calibration.

We claim three things about $s(x)$:

  1. It is larger for member items than for non-members.
  2. The gap survives clean-data GRPO far better than static loss gaps do, because RL suppresses the memorized behavior without erasing the nearby weights.
  3. It rises with the measured score inflation on each benchmark.

Nothing in the supplied literature measures contamination through adaptation dynamics. DyVal and PromptBench perturb inputs to a fixed checkpoint, CAP and Self-Critique probe behavior at inference, and ConStat compares aggregate scores across models.

Research Program

The probe uses a two-fold split. Candidate items are divided in half. We fit a rank-8 adapter on one half for $k \in \{1, 4, 16\}$ steps at a fixed small learning rate, then score savings on the other half, and swap. Every item therefore gets a score from one or two cheap adaptation runs rather than one run per item. The same adapter recipe, including seeds, is applied to the reference model. Following Meta-Adapters, we also test whether meta-learning the adapter's learning rate on known-clean benchmarks sharpens the signal. We treat this only as an optional refinement, since the core claim must hold with plain fine-tuning.

We will study three contamination regimes, taken from the cited papers:

  1. Label-level contamination followed by clean GRPO, which is the Fragility Stage I setup.
  2. Chain-of-thought contamination of a reasoning model, which is Stage II.
  3. Soft contamination by semantic duplicates, following Soft Contamination Means Benchmarks Test Shallow Generalization.

The soft regime is the hard case for item-level detection. That paper finds roughly 20% gains on both seen and unseen items of the same benchmark, so savings may leak onto non-members as well. We therefore report both item-level AUROC and a benchmark-level test: whether the mean savings on the suspected benchmark exceeds the mean savings on a structurally matched fresh set. For formal tasks, the fresh set is DyVal-generated items (DyVal). For other tasks, it is post-cutoff items in the style of LiveBench.

One limitation is stated up front. The probe needs gradient access, whether through open weights or a fine-tuning API. It therefore does not replace black-box audits of closed models.

Baselines

The naive reference point is the static loss and Min-K% probability of the original model on each item. The RL papers report these near chance after post-training: 0.42 to 0.53 AUC on RL-MIA, and about 50% AUROC for reference-free methods after GRPO in the Fragility setup. Any useful detector must clear this by a wide margin.

The strongest static competitor is LiRA from On The Fragility of Benchmark Contamination Detection in Reasoning Models. It reaches 89.1% AUROC on SFT-contaminated Qwen2.5-7B-Instruct and loses about 14 points after one epoch of GRPO. We run it with the same reference model $\phi$ we use for calibration, so both methods have identical reference information. Beating it means matching it within a few points before GRPO and exceeding it by at least ten AUROC points after GRPO.

The strongest behavioral competitor is Self-Critique from Detecting Data Contamination from Reinforcement Learning Post-training. It reports 0.70 average AUC on Qwen2.5-7B-Instruct and 0.64 on DeepSeek-Math-7B-Instruct on RL-MIA. We run it on the same models and items with its published sampling settings, and we match its compute to ours by counting forward and backward token passes. Beating it on RL-MIA means a clear AUC margin over 0.70 on Qwen2.5-7B-Instruct at no more than twice its compute.

We also run CAP's perturbation consistency (CAP). The file gives no AUROC for it on reasoning benchmarks, so there is no reported result on Lacuna for CAP on AIME-style tasks, and we will measure it ourselves.

For benchmark-level claims, the strongest competitor is ConStat (ConStat). It flags 89% of syntax-contaminated and 98% of sample-contaminated models by comparing scores against a curve fitted on reference models. We give it the same reference fleet we use.

Two ablations isolate the key ingredient. The first replaces the benchmark-sibling support set with an equal-size set of unrelated clean math data. If savings then separates members equally well, the effect is just undoing RL globally, not reactivating a benchmark-specific trace. The second drops the reference-model subtraction to show how much the calibration contributes.

Why not just…?

Why not just use ConStat, which is performance-based and should not care about RL cloaking? This is a real threat at the benchmark level. ConStat measures inflated scores directly, and inflation survives RL. Its limits are that it answers a model-level question, requires rephrased or synthetic reference benchmarks, and relies on a fleet of reference models. Soft contamination may also inflate rephrased references, which would shrink the gap ConStat measures. We will run both on the soft-contamination regime. If ConStat matches our benchmark-level test there, our contribution narrows to item-level detection, and we will say so.

Why not just fine-tune the model on clean data and then run LiRA? This "de-cloaking" alternative may partly work, and it is exactly what the unrelated-support ablation tests. Our hypothesis predicts that sibling-item support recovers much more separation than generic support, because the gain comes from aligned gradients on a shared memorized trace. If generic support does just as well, the simpler recipe wins, and the paper becomes a demonstration of it.

Why not just use behavioral probes like Self-Critique or CAP, which need no gradients? They work black-box, which our probe cannot. However, both read the same collapsed output distribution that RL reshapes, and Self-Critique's best reported AUC is 0.70. The Stage II chain-of-thought setting is the discriminating experiment. There, the Fragility paper finds that the model generalizes its confidence to unseen items. That should defeat entropy-based probes but not a relearning test calibrated against a reference model.

Proposed Experiments

The first experiment reproduces Fragility Stage I on Qwen2.5-7B-Instruct. We contaminate half of the AIME, AMC, OlympiadBench, Minerva Math and GPQA Diamond items by supervised fine-tuning, then run one epoch of clean GRPO. We report AUROC and true-positive rate at 5% false-positive rate for all detectors before and after GRPO, with the clean Qwen2.5-7B-Instruct as $\phi$.

The second experiment repeats this with chain-of-thought contamination of DeepSeek-R1-Distill-Llama-8B, where all ten reported detectors sit at 50 to 58%.

The third runs the probe on RL-MIA against Self-Critique's reported 0.70 and 0.64.

The fourth builds soft contamination on MuSR by fine-tuning on semantic duplicates at the realistic 4-in-10,000 ratio. Here we report both item-level and benchmark-level savings against a matched DyVal or post-cutoff fresh set.

Across all regimes, we correlate each benchmark's mean savings with its measured pass@1 inflation.

The decisive check is simple. If savings AUROC after GRPO stays within about five points of its pre-GRPO value while LiRA loses about fourteen, and the sibling-support probe clearly beats the unrelated-support ablation, then relearning speed is a contamination signal that RL does not erase. If savings collapses alongside LiRA, the hypothesis that RL suppresses but does not erase memorized weights is wrong, and that negative result is itself informative for the cloaking analysis in the Fragility paper.