Catching Rephrased Benchmark Leaks by Asking Whether Old Answers Stick

When a language model has seen a test set during training, its benchmark score partly measures memory rather than skill. This is called benchmark contamination. Current detectors mostly look for traces of the exact test text: repeated word sequences, unusually low surprise on specific tokens, or a preference for the order in which the dataset was published. Evading Data Contamination Detection for Language Models is (too) Easy shows how fragile this is. If a model is trained on GPT-4 paraphrases of the test questions instead of the originals, it keeps score gains of up to 15% on MMLU, GSM8K, TruthfulQA and ARC-Challenge. Meanwhile, every sample-level detector they tried falls to roughly 1% true positive rate at a 1% false positive rate. In that setting the text-level trace is gone, but the leaked answer is not.

We propose to detect that leaked answer directly. For each benchmark item, we build verified counterfactual variants: small edits that keep the scenario but change the correct answer, such as altering a quantity in a GSM8K problem or flipping a premise in an ARC question. A model that reasons will usually move to the new answer. A model that has memorized the content-to-answer link will tend to give the original gold answer, which is now wrong. We turn this "stale answer" behavior, and a few related black-box contrasts, into a small set of engineered features and feed them to a shallow classifier. The claim is specific: under paraphrase-based contamination, item-level detection becomes possible again from text outputs alone, where existing detectors are at about 1% TPR at 1% FPR.

Core Claim

Paraphrase-based contamination changes where the evidence of memorization lives. The token-likelihood signals that Detecting Pretraining Data from Large Language Models exploits average the log-probability of the least likely tokens (Min-K% Prob). They rely on the model having seen the exact string. Paraphrase training removes that dependence. What it cannot remove is the association between a problem's situation and its answer, because that association is exactly what produces the inflated score.

The central feature is a calibrated stale-answer rate. For a model $M$ and item $i$ with gold answer $a_i$, let $c_{i1},\dots,c_{iK}$ be $K$ verified counterfactual edits whose correct answers all differ from $a_i$. Define

$$S_M(i) = \frac{1}{K}\sum_{k=1}^{K} \mathbb{1}[M(c_{ik}) = a_i] \;-\; \bar{S}_{\text{ref}}(i),$$

where $M(c)$ is the model's answer to variant $c$, and $\bar{S}_{\text{ref}}(i)$ is the same rate averaged over clean reference models of similar overall accuracy. The subtraction separates "answers this kind of edit wrongly, as weak models do" from "answers it with the specific old answer." This borrows the hardness-correction idea from ConStat at the level of a single item.

We pair $S_M(i)$ with a handful of other output-only contrasts and give them to logistic regression and a shallow gradient-boosted tree:

  • agreement across repeated samples on the original item versus on its counterfactuals, which serves as a confidence asymmetry;
  • accuracy on meaning-preserving rephrasings, which is CAP's consistency ratio;
  • whether the model's reasoning chain on a counterfactual uses the original's quantities.

The feature-engineering bet is the one documented in MaxCutBench and Semantic Linking in Convolutional Neural Networks for Answer Sentence Selection. Once the inputs encode the right structure, here a deliberate contrast between original and edited items, a simple learner does as well as a heavy one.

Relation to Prior Work

The closest work is RADAR. It already frames contamination detection as a shallow classifier over 37 engineered features and reports 93.0% accuracy on its own 100-item test set. RADAR differs from this proposal in three ways. Its features need hidden states and attention, so it cannot audit API models. It was evaluated on verbatim recall-versus-reasoning cases and never against paraphrase-based contamination. Its features describe how the model computes on the item as given, not how its answer responds to edits that should change it. We reuse RADAR's shallow-ensemble recipe and treat its feature set as a white-box comparison.

Two black-box methods supply ingredients. CAP measures how performance holds up under meaning-preserving perturbations and thresholds a single consistency ratio. Under paraphrase-based contamination that ratio may stay near one, because the model was trained on paraphrases, so we keep it only as one feature. ConStat defines contamination as performance that fails to generalize to reference benchmarks, corrected for difficulty using about 20 clean models. It reports detecting 89% of syntax-contaminated and 98% of sample-contaminated models. It works at the level of a whole model and benchmark, and it constrains our design: any item-level signal must beat or complement it once aggregated. Data Contamination Quiz tests whether the model recognizes the original wording among perturbed versions. That is precisely the cue paraphrase training removes.

Paraphrase-based contamination itself, from Evading Data Contamination Detection, is the motivating negative result and the attack we reproduce. Proving Test Set Contamination in Black-Box Language Models relies on canonical ordering and so does not apply once items are paraphrased and shuffled.

Research Program

Controlled contamination suite. Following the paraphrase-attack protocol, we fine-tune open models, starting with the Llama-2 and Mistral-7B families used across these papers, on OpenOrca as clean background data. Into this we inject GPT-4 paraphrases of a random half of the items from GSM8K, ARC-Challenge and MMLU. A verbatim-injection condition and a clean condition serve as controls. This gives ground-truth item-level membership, which no public benchmark in the file provides for paraphrase-based contamination.

Counterfactual construction. For GSM8K, the step-by-step solutions let us change the numbers and recompute the answer programmatically, so these variants are verified exactly. For ARC-Challenge and MMLU, counterfactuals are generated by an LLM and kept only if an independent model and a human spot-check agree that the gold option changed. We expect this filter to leave only a subset of items, and we report coverage honestly.

Classifier and transfer. Features are extracted from sampled text outputs only. The shallow classifiers are trained on contaminated and clean models from one family and tested on another without retraining. This also answers the open transfer question RADAR leaves.

Adaptive attacker. We also run an attacker who trains on counterfactual variants with their correct answers as well. This is the natural countermeasure, and it partly converts memorization into genuine learning of the task. We measure two things together: detection under this attacker, and the attacker's score gain on fresh, never-leaked items. If detection fails only when the score inflation also disappears, the method has done its job.

Baselines

Do-nothing and naive references. The do-nothing reference is the chance detector, with TPR equal to FPR. The naive reference is 13-gram overlap plus whole-text perplexity thresholding. This is what a practitioner would run first, and it fixes the floor that paraphrase-based contamination is known to push toward.

Strongest likelihood-based detector. This is Min-K% Prob from Detecting Pretraining Data from Large Language Models. It reports a 7.4% AUC gain over the previous best baseline on WikiMIA. It is the most widely used reference-free item-level signal, and it is run with the same models and with logprob access, which favors it. On our exact question, the strongest reported number is the paraphrase-attack paper's: existing sample-level detectors reach approximately 1% TPR at 1% FPR on paraphrase-contaminated MMLU, GSM8K, TruthfulQA and ARC-Challenge (Evading Data Contamination Detection). Beating it means a clearly higher TPR at 1% FPR on the same attack. Our target is above 30% on GSM8K, where counterfactuals are exact.

Stacked existing scores. This is the obvious combination of the two research directions. Min-K%, CAP's consistency ratio and the quiz's selection rate are put into the same logistic regression with the same training data. It isolates whether the gain comes from multivariate stacking or from the new stale-answer feature.

Benchmark-level comparison. ConStat is run with its released reference-model protocol. Our item scores are averaged into an estimate of benchmark inflation and compared against ConStat's δ and its detection rate across contaminated and clean models. ConStat's reported model-level detection rates (89% and 98%) were not measured under the paraphrase attack. For that setting, we found no reported result on Lacuna for ConStat.

White-box reference. RADAR-style internal features plus our features, trained identically, show how much access to internals adds. For RADAR under paraphrase-based contamination, we found no reported result on Lacuna.

Ablations and fairness. The key ablation removes $S_M(i)$, and separately removes its reference calibration. All learned baselines get the same labeled training models, the same hyperparameter search budget and the same number of model queries per item.

Why not just…?

Why not just stack the existing detector scores? This is the most natural combination, and we run it as a baseline. Every one of its inputs keys on the original wording or on consistency under meaning-preserving edits. Paraphrase training is designed to defeat the first and plausibly leaves the second intact. If the stacked model matches ours, the stale-answer feature is not needed and we will say so.

Why not just use ConStat? ConStat is a real threat if the question is only "is this model's benchmark score inflated." Its definition of contamination as non-generalizing performance should in principle survive paraphrasing. What it does not give is which items are leaked, which is needed to report a cleaned score or to audit a single evaluation. It also needs a full reference benchmark per task. Our aggregation experiment decides the question. If item-level stale-answer scores, once averaged, track the true injected fraction no better than ConStat's δ, our contribution narrows to item-level localization.

Why not just train a deep detector on raw outputs or embeddings? The feature-engineering literature in the file, including MaxCutBench, suggests depth adds little once the inputs encode the right contrast. More importantly, labeled contaminated models are scarce, so a deep detector is likely to overfit to model-family quirks. We test this directly. A fine-tuned small encoder given the concatenated original and counterfactual outputs is trained under the same budget and compared on cross-family transfer.

Why not just move to live benchmarks? LiveBench and similar efforts avoid leakage until their items are crawled. They do not help audit the static benchmarks still used in most papers. Our method is complementary rather than a replacement.

Proposed Experiments

The decisive check is TPR at 1% FPR, plus AUC, for item-level membership on paraphrase-contaminated GSM8K, with Llama models used for training and Mistral models for testing. Success means a large gain over the roughly 1% reported for existing detectors, and the gain disappearing when the stale-answer feature is ablated. Secondary checks repeat this on the verified counterfactual subsets of ARC-Challenge and MMLU, where we expect weaker but nonzero signal.

The clean-model false-positive analysis matters most for credibility. Clean models that make systematic errors could produce "stale" answers by coincidence, especially on multiple choice. The reference calibration in $S_M(i)$ must keep the false positive rate at its nominal level on held-out clean models.

Finally, under the adaptive attacker we report detection rate against the remaining score inflation on fresh items. The method fails if inflation persists while detection collapses.