When a language model scores well on a public benchmark, we usually cannot tell whether it solved the problems or had already seen them, or close rewrites of them, during training. The cheap checks look at how probable the model finds the question text. Those checks fail once the leaked data is paraphrased: fine-tuning on rewritten GSM8k problems raises benchmark scores by about 10 points while perplexity and lowest-token-probability scores stay at chance (DICE). Methods that do catch paraphrased leakage read the model's internal activations. They then train a supervised classifier, which needs labeled contaminated examples and, for DICE, an uncontaminated copy of the same model. An auditor handed a single released model has neither.
We propose a different, label-free test drawn from anomaly detection. That field finds suspicious inputs by where they sit in the geometry of a network's hidden representations, rather than by what the network outputs. Training on an item, or on many rewrites of it, should turn that region of hidden space into a local peak of density: slightly different versions of the question get pulled together into a tight, low-dimensional cluster. We therefore perturb each benchmark item a few ways, embed the variants at an intermediate layer, and ask whether the model collapses them together more than it collapses comparable variants of text it cannot have seen. The claim is that this per-item "neighborhood collapse" detects paraphrased contamination that output-probability methods miss. It needs no clean twin model, no labels, and no clean twin benchmark.
Core Claim
Maximum-likelihood training turns training points into local modes of the model's distribution. Min-K%++ builds on this, but it can only measure it in the discrete space of next-token probabilities, and its authors note that the underlying argument is about continuous distributions. Hidden states are continuous. Manifold-based anomaly detection has shown in vision that density in a compressed latent layer is more reliable than surface signals. Kernel density on final hidden layers flags off-manifold adversarial points (Towards Robust Detection of Adversarial Examples), and latent density beats reconstruction error for spotting novel driving scenarios (Anomaly Detection in Multi-Agent Trajectories).
We turn that logic around. Contaminated items are not off-manifold. They are too tightly on it. For a benchmark item $x$ with meaning-preserving variants $x_1,\dots,x_m$ (paraphrases, renamed entities, changed numbers), let $h_\ell(\cdot)$ be the mean-pooled hidden state at layer $\ell$ and $e(\cdot)$ the input-embedding-layer representation. The collapse score is
$$ C_\ell(x) = \frac{\operatorname{median}_i \lVert h_\ell(x_i) - h_\ell(x)\rVert}{\operatorname{median}_i \lVert e(x_i) - e(x)\rVert}, $$
which is the ratio of how far apart the variants are after processing to how far apart they were going in. We z-score $C_\ell(x)$ against the same statistic computed on variants of reference text that postdates the model's training cutoff, playing the role that the vocabulary mean and variance play in Min-K%++. A second score is the TwoNN intrinsic dimension of the variant cloud. TwoNN estimates how many directions a point cloud actually spreads along from the ratio of each point's first and second nearest-neighbor distances. We predict that contaminated items show strongly negative z-scores and reduced intrinsic dimension, and that clean items do not.
This differs from each of the three closest papers in a specific way. DICE chooses its layer by comparing the audited model against a clean reference model and then trains a supervised MLP; we use neither. RADAR trains classifiers on 37 hand-built features from 30 labeled examples; we fit nothing on labels. Min-K%++ stays at the output logits; we measure geometry inside the network.
Research Program
The work has three parts.
The first is layer selection without a reference model. The geometry of hidden representations of large transformer models found that intrinsic dimension across layers follows the same three-phase arc in many models: it rises, falls to a minimum at an intermediate "semantic bottleneck," and rises again toward the output. Representations at the minimum carry the most semantic content. We choose $\ell$ at that minimum, using only generic text. This creates a tension to resolve: DICE found contamination sensitivity in late layers (layer 29–30 of 33), not at the bottleneck. We therefore report the collapse score at every layer. We also test an aggregate across layers, in the spirit of Deep neural rejection, which showed that monitoring several depths catches anomalies a final-layer detector misses.
The second is perturbation design. Variants must change the surface form, so the input-embedding distance is meaningful, while keeping the problem's meaning. For math we use numeric substitution in the style of GSM-hard, entity renaming, and LLM paraphrase. We verify the resulting spread in input-embedding distance and check that the score does not simply track paraphrase quality.
The third is a stress test against adaptive evasion. Obfuscated Activations Bypass LLM Latent-Space Defenses drove Gaussian Mahalanobis and probe monitors from about 100% recall to 0%. The same paper found an "obfuscation tax": forcing activations along benign-looking paths degraded reasoning-heavy tasks, with SQL accuracy falling toward zero. We ask whether a contaminator who fine-tunes with a penalty on collapse can keep the benchmark gain. If hiding the collapse costs the gain, the score is informative even against a motivated adversary.
Baselines
The naive reference points are perplexity and Min-k% Prob computed on the question text. DICE reports both at coin-flip level on paraphrased GSM contamination, and we expect the same. They are run on the identical items and models as our score.
The strongest reference-free competitor is Min-K%++, which is the output-level version of the same "training points are modes" argument. Its paper reports gains of 6.2 to 10.5 AUROC points over Min-K% on WikiMIA, and parity with reference-model methods on MIMIR. The file gives no absolute AUROC for either benchmark, so the target on those is to match or exceed our own re-run of Min-K%++. On paraphrased contamination we expect to beat it clearly.
The strongest perturbation-based competitor is Polarized Augment Calibration (PAC), which perturbs samples and compares the model's output token certainty. PAC reports +4.5 AUC on WikiMIA and +5.9 on StackMIA over prior methods. The key ablation uses exactly our perturbation sets but scores them in logit space: we compute Min-K%++ and PAC-style certainty on the variants. This isolates whether hidden-state geometry adds anything beyond the perturbations themselves.
A second ablation replaces the collapse score with plain Mahalanobis distance of $h_\ell(x)$ from a Gaussian fitted on reference text, without perturbations. This tests whether the neighborhood structure matters or ordinary latent out-of-distribution scoring suffices.
The supervised upper bound is DICE, run with its clean reference model and its labeled MLP. It reports AUROC near 100.0 on in-distribution contamination and wild-model correlation of $R^2$ between 0.61 and 0.75. Our success criterion is not to beat DICE. It is to come within a few AUROC points of it while using no reference model and no labels.
All methods get the same audited models, the same forward-pass budget per item (Min-K%++ and PAC are allowed $m$ extra passes), and hyperparameters (DICE's layer and MLP, our $m$ and $\ell$) tuned on a shared held-out validation split.
Why not just…?
Why not just compare accuracy on GSM8k against a clean twin like GSM1k? A Careful Examination of LLM Performance on Grade School Arithmetic does this and finds drops of up to 8% for some model families. The approach works, but it answers a different question. It gives a population-level gap, it depends on expensive human-matched twins that leak once released, and it cannot say which items are contaminated. We treat the gap as external validation rather than a competitor: across open models, our mean collapse score should predict the GSM8k–GSM1k gap. We compare that correlation with the reported Spearman $r^2 = 0.36$ for GSM8k log-likelihood.
Why not just run Min-K%++ or PAC on the paraphrases? This is the real threat, because the perturbations may carry most of the signal. The logit-space ablation settles it. If logit scores on the same variant sets match our hidden-state score on paraphrased contamination, we will report that the geometry adds nothing. DICE's finding that output-probability methods sit at chance on in-distribution data is the reason we expect otherwise.
Why not just use DICE with the model's public base checkpoint as the clean reference? Sometimes a base checkpoint exists. But contamination during pretraining lives in the base checkpoint itself, and DICE still needs labeled contaminated examples to train its probe. We run DICE wherever a base checkpoint is available and report the gap between it and our score honestly.
Proposed Experiments
The controlled experiment follows DICE's recipe. We fine-tune an open model on OpenOrca mixed with 2% contamination, taken from one random half of the GSM8k test set. We run it in three variants: verbatim, GSM-i-Syn-style paraphrases, and numeric rewrites. The other half is held out. That split gives item-level ground truth inside a single model, and we report AUROC for contaminated versus held-out items. A clean OpenOrca-only model serves as a negative control; its two halves should be indistinguishable. We vary contamination to 1–2 exposures, where Min-K%++ notes that rare-sequence memorization evades probability thresholds.
For pretraining leakage we evaluate on WikiMIA and MIMIR. For wild validation we correlate model-level scores with the GSM8k–GSM1k gap and with the order-based test of Proving Test Set Contamination. That paper reports $p = 0.001$ for Mistral-7B on AI2-ARC, which gives us a case to check for agreement.
The decisive checks are three. First, the collapse score must stay near DICE's AUROC on paraphrased contamination, where perplexity and Min-k% fail. Second, the hidden-state score must beat the logit-space ablation on the same variants. Third, under adaptive fine-tuning against the score, either the contamination remains detectable or suppressing the collapse erases most of the 10-point benchmark gain. Failing the second check would mean the geometry is not the active ingredient. Failing the third would mean the score is not robust to a motivated contaminator.
References
- DICE: Detecting In-distribution Contamination in LLM's Fine-tuning Phase for Math Reasoning
- Min-K%++: Improved Baseline for Pre-Training Data Detection from Large Language Models
- Towards Robust Detection of Adversarial Examples
- Anomaly Detection in Multi-Agent Trajectories for Automated Driving
- RADAR: Mechanistic Pathways for Detecting Data Contamination in LLM Evaluation
- The geometry of hidden representations of large transformer models
- Deep neural rejection against adversarial examples
- Obfuscated Activations Bypass LLM Latent-Space Defenses
- Data Contamination Calibration for Black-box LLMs
- A Careful Examination of Large Language Model Performance on Grade School Arithmetic
- Proving Test Set Contamination in Black-Box Language Models