Large language models are often scored on benchmarks whose questions leaked into their training data. The usual tests for this leakage ask whether the model is unusually confident on a test item, for example by checking whether its loss or token probabilities are suspiciously good. Recent work shows these tests stop working once a model is post-trained with reinforcement learning. After that training, contaminated and clean items look statistically the same at the output.
We propose a different test. Instead of reading the model's confidence on the item itself, we slide the model's input along a continuous path that ends at the benchmark question, and we watch how sharply the model's internal state changes along the way. A model that has memorized an item should react abruptly only in a small neighbourhood of the exact wording. A model that genuinely solves the problem should respond smoothly along the whole path.
The opportunity is concrete. On The Fragility of Benchmark Contamination Detection in Reasoning Models contaminated reasoning models on AIME, AMC, GPQA Diamond, OlympiadBench and Minerva Math, then post-trained them with GRPO (a reinforcement-learning method with clipped policy updates). After a single GRPO epoch, reference-free detectors fell to about 50% AUROC. When contamination arrived as chain-of-thought fine-tuning of an already-trained reasoning model, all ten detectors tested averaged 50–58% AUROC, while the model still gained 11.7% on the benchmark.
A separate result points to where a signal might survive. Memorization Detection in Diffusion Models via Text Embedding Interpolation interpolates a text-to-image model's prompt embedding from zero to the target prompt. It finds that for memorized prompts the model's sensitivity to its conditioning stays flat for most of the path and spikes near the end. Measuring that spike yields AUC 0.998 on Stable Diffusion v1.4. Nobody has asked whether the same spike exists in autoregressive language models, or whether it survives the reinforcement learning that erases output-level signals.
Core Claim
Memorized benchmark items leave a localized spike in the input-embedding Jacobian of a language model. The Jacobian here means how much the model's hidden state moves per unit change of its input embeddings. The spike appears along a continuous path that ends at the verbatim item.
We claim this spike persists after GRPO for data seen during fine-tuning, because RL reshapes output probabilities more than it reshapes input sensitivity. The claim is falsifiable, and one outcome of it is unknown. For chain-of-thought contamination, the Fragility authors argue the model "internalizes the reasoning process". On the Geometry of Memorization: Interpolation and Second-Order Representation Irregularity proves, for small networks, that rule-like generalization spreads curvature smoothly while fitting specific targets concentrates it. If the internalization account is right, the spike should vanish in that setting. Establishing which regime holds is itself a result: it tells the field whether chain-of-thought contamination is detectable from the model's weights at all.
Why Continuous Paths Are Available in Language Models
The method assumes a language model behaves sensibly on inputs between real token embeddings. Language Models Are Implicitly Continuous supplies that premise. It feeds Llama 3, Mistral and Gemma linear blends of token embeddings and finds that about 80% of outputs stay within the logical bounds of the two endpoints. That is enough to treat the embedding path as a meaningful probe rather than an off-manifold artifact.
The same paper constrains us in two ways. The remaining 20% of cases mean some paths will cross regions where the model behaves erratically. And its admitted mechanistic opacity means we cannot assume where along the path any irregularity will sit.
Research Program
We define two paths into each benchmark item. Write $e_1$ for the sequence of input embeddings of the verbatim question and $e_0$ for a start point. The path is $e(\lambda) = (1-\lambda)e_0 + \lambda e_1$, where $\lambda$ runs from 0 to 1.
The first path is the direct analogue of the diffusion work. Here $e_0$ is a null input: the same positions filled with the embedding of a padding or neutral token.
The second path is a counterfactual path designed to separate "this wording" from "this problem". Here $e_0$ is a length-matched surface variant of the same question: variables renamed, entity names swapped, function words substituted with same-length tokens. Every position aligns with $e_1$ and the answer does not change. A model that reasons should be nearly indifferent along this path. A model that memorized the string should not be.
Along each path we measure a sensitivity score:
$$s(\lambda) = \lVert \partial h(e(\lambda)) / \partial e \rVert_F$$
Here $h$ is the final-layer hidden state at the position where the model begins its answer, and $\lVert\cdot\rVert_F$ is the Frobenius norm. We estimate this norm with a Hutchinson estimator (averaging squared Jacobian–vector products over random directions), exactly as the diffusion paper does, so no full Jacobian is ever formed.
We summarize each curve with two numbers. The first is the ratio of the maximum of $s$ over $\lambda \in [0.8, 1]$ to its mean over the whole path; we take this window because the diffusion paper places local memorization spikes near $\lambda \approx 0.8$. The second is a localization score modelled on the curvature localization measure $L(\theta)$ of the geometry-of-memorization paper, computed from finite differences of $s$. Both are unsupervised and need no reference model.
The engineering piece is deliberately modest. Existing evaluation systems pass only text between stages. A Composable Evaluation System for Reproducible Omni-Modal Foundation Model Evaluation routes predictions through an intermediate schema across data iteration, inference, post-processing and metric computation. PromptBench: A Unified Library for Evaluation of Large Language Models modularizes discrete prompt attacks and DyVal. Neither exposes embeddings or Jacobian–vector products.
We will add a white-box record type to that schema: embeddings, path coordinates, and sensitivity estimates. With it, the detector and all baselines run as interchangeable metric modules over one set of forward passes. This is what makes the comparisons below fair. Every detector sees identical items, identical models and identical seeds, following the reproducibility discipline of DeepDIVA: A Highly-Functional Python Framework for Reproducible Experiments. The limitation is real: the method requires open weights, and the schema cannot extend it to API-only models.
Baselines
The naive reference point is chance, 50% AUROC. This is where the Fragility paper places reference-free detectors after one GRPO epoch, and roughly where all ten of its detectors land under chain-of-thought contamination (50–58%). Any method that does not clear this band in the post-RL settings has failed.
The strongest existing competitor is LiRA, a membership-inference attack that compares the target model's likelihood against reference models. The Fragility paper reports 89.1% AUROC for LiRA on supervised-contaminated models before RL, falling by roughly 14% after one GRPO epoch. We will also run the loss and Min-K% detectors from the same paper, which lose roughly 16% under GRPO. All three will run on the exact contaminated checkpoints we produce, with the same reference models and the same held-out clean items. Each gets the same hyperparameter search budget as our two path summaries.
Beating LiRA means two things. After GRPO, we must exceed its AUROC by at least five points while losing less than five points relative to our own pre-RL score. Under chain-of-thought contamination, we must reach a mean AUROC of 0.70 or higher.
The black-box perturbation competitor is PEARL from Memorization or Interpolation ? Detecting LLM Memorization through Input Perturbation Analysis. It flips bits of a binary text encoding and flags items whose performance collapses. We will run it with its sensitivity threshold tuned on the same validation split we use for our cutoffs.
We also run a behavioral competitor: the accuracy drop between the original items and paraphrased, option-permuted versions, in the style of Dynamic Evaluation of Large Language Models by Meta Probing Agents. For this we use an open model as the rewriter, so cost is comparable.
Three ablations isolate what the path contributes. The endpoint-only ablation computes $s(1)$ alone, with no path. The isotropic-noise ablation replaces the path with random Gaussian perturbations of $e_1$ of matched norm and measures variance of $h$. The path-type ablation compares the null path with the counterfactual path. Finally, a linear probe trained on hidden states with contamination labels serves as a supervised upper bound. It is not a deployable competitor, since real audits have no labels.
Why not just…?
Why not just use loss, Min-K% or LiRA, which need no gradients? Because the Fragility study shows these are exactly the signals that GRPO removes, driving reference-free methods to about 50% AUROC and all ten detectors to 50–58% under chain-of-thought contamination. The open question is whether anything survives. We will report these detectors on our own checkpoints, so the collapse is reproduced rather than assumed. If they do not collapse in our replication, our motivation weakens, and we will say so.
Why not just paraphrase the benchmark and measure the score drop? Paraphrase drops are informative. The Meta Probing Agents paper reports a 15.7% accuracy fall for GPT-4-Turbo on fully probed MMLU. But this signal gives per-benchmark evidence, not per-item evidence. It is also confounded by paraphrases that are simply harder. And the Fragility authors argue it fails precisely when the model has internalized the reasoning, since rephrased items remain solvable. Our counterfactual path measures the same "wording versus problem" contrast inside the model, item by item, without a judge model. The behavioral baseline stays in the comparison. If it matches our detector in per-item AUROC, the geometric machinery is unnecessary.
Why not just take the Jacobian at the question itself, or add random noise, with no path? This is the most serious threat. In the diffusion setting the spike is near the endpoint, so a pointwise measurement might capture most of the signal. The diffusion paper reports that pointwise score differences are noisy and need averaging across seeds, but that is a different model family. The endpoint-only and isotropic-noise ablations settle this directly. If either comes within two AUROC points of the path-based score, we will drop the path and report the simpler detector as the contribution.
Why not just train a probe on hidden states? A probe needs contamination labels. Real audits lack them, and a probe fit on one contamination recipe may not transfer to another. We train it only as an upper bound and test its transfer from Stage I checkpoints to Stage II checkpoints. If the unsupervised path score approaches that bound while the probe fails to transfer, that is the practical argument for it.
Proposed Experiments
The primary experiment reproduces both contamination regimes of the Fragility paper on AIME, AMC, GPQA Diamond, OlympiadBench and Minerva Math. For Stage I, we fine-tune an open base model on half of each benchmark and then apply GRPO. For Stage II, we fine-tune DeepSeek-R1-Distill-Llama-8B on chain-of-thought solutions for half of each benchmark. The other half of each benchmark serves as clean negatives.
We compute AUROC and true-positive rate at 1% false-positive rate for every detector at three checkpoints: before RL, after one GRPO epoch, and after three epochs. We also record how the full $s(\lambda)$ curves change over training. This lets us observe whether the spike fades gradually or survives intact.
Before any of this, a sanity check reruns the Jacobian detector on Stable Diffusion v1.4 with the published memorized prompts. The target is the reported 0.998 AUC, which confirms our estimator is implemented correctly before we change model family.
The decisive check is the Stage II result, read against the counterfactual path. Suppose the spike persists there while LiRA sits near chance. Then contamination through reasoning traces is still detectable from weights, and the "internalization" account is at best partial. Suppose instead the spike vanishes and the curves become as smooth as clean items. That would be the first representation-level evidence that such contamination is genuinely indistinguishable from generalization, which is the regime the geometry-of-memorization theory predicts for rule learning.
A secondary, label-free test applies the detector to public open-weights models. The first input is MMLU items, which Measuring Massive Multitask Language Understanding released years ago and which are plausibly in pretraining data. The second is post-cutoff LiveBench items, which LiveBench: A Challenging, Contamination-Limited LLM Benchmark builds from sources released after mid-2024. We have no per-item ground truth for these models. The chronological prediction is therefore only directional: spike ratios should be higher on MMLU than on LiveBench for models trained before LiveBench's sources existed.
The final check uses DyVal problems from DyVal: Dynamic Evaluation of Large Language Models for Reasoning Tasks, which are generated fresh and therefore uncontaminated by construction. Scores on these items give a clean-item false-positive rate across difficulty levels D1–D4. Harder problems should not look "memorized" merely because the model is less stable on them.
References
- On The Fragility of Benchmark Contamination Detection in Reasoning Models
- Memorization Detection in Diffusion Models via Text Embedding Interpolation
- On the Geometry of Memorization: Interpolation and Second-Order Representation Irregularity
- Language Models Are Implicitly Continuous
- A Composable Evaluation System for Reproducible Omni-Modal Foundation Model Evaluation
- PromptBench: A Unified Library for Evaluation of Large Language Models
- DeepDIVA: A Highly-Functional Python Framework for Reproducible Experiments
- Memorization or Interpolation ? Detecting LLM Memorization through Input Perturbation Analysis
- Dynamic Evaluation of Large Language Models by Meta Probing Agents
- Measuring Massive Multitask Language Understanding
- LiveBench: A Challenging, Contamination-Limited LLM Benchmark
- DyVal: Dynamic Evaluation of Large Language Models for Reasoning Tasks