Maximum-Entropy Sibling Nulls for Auditing Contaminated Reasoning Models

When a reasoning model has seen a benchmark question during training, the usual giveaway is gone: reinforcement learning flattens the gap in likelihood between seen and unseen questions, and today's detectors fall to coin-flip accuracy on these models. This proposal stops asking "is this question unusually likely under the model?" and asks "does the model behave unusually well on this exact question compared with near-identical copies of it?" For each benchmark item we generate a handful of "siblings": the same problem with new numbers, reordered options, or reworded text, each with a verified answer. We sample the model on the original and on its siblings. We then build the least-committed probability model of the sibling behavior, using the maximum entropy principle (pick the flattest distribution that still matches a few measured averages). The original's surprise under that null model becomes a calibrated, item-level p-value for contamination.

This matters because static benchmarks such as AIME, GPQA and MMLU remain the currency of capability claims, and auditing them now requires strict false-positive control rather than a decent average ranking. The maximum entropy principle is not decoration here. With only four to eight siblings per item, and with behavioral statistics that are bounded and skewed (accuracy near 0 or 1, edit distances in [0,1]), a Gaussian z-score gives wrong tail probabilities and a raw rank test cannot produce small p-values. Matching a few moments under maximum entropy gives the smoothest tail extrapolation the data allow, and the experiments test that claim directly.

Core Claim

Reinforcement learning hides contamination in absolute likelihoods but not in item-specific behavior. RL with verifiable rewards collapses the diversity of reasoning paths on prompts it trained on (Detecting RLVR Training Data via Structural Convergence of Reasoning). Memorization of a specific item's answer or derivation should not transfer to a re-instantiated sibling whose answer differs. Genuine generalization from chain-of-thought exposure, which On The Fragility of Benchmark Contamination Detection in Reasoning Models identifies as a reason detectors fail, should lift the siblings too. A sibling-relative test therefore separates two things the field currently conflates: an item whose score is inflated specifically, and an item family the model has actually learned to solve. We claim that a maximum-entropy null fit to sibling statistics detects the first with controlled false-positive rates where absolute detectors fail, and leaves the second alone.

How the Pieces Fit, and What Is Already Done

The motivating evidence comes from two papers that point in opposite directions. The fragility paper shows that clipping in PPO-style RL compresses log-probability gaps. Ten detectors sit at 50–58% AUROC on a distilled reasoning model, and none of them is replaced. The structural-convergence paper finds a surviving signal in rollout diversity. Its Min-kNN distance averages the smallest nearest-neighbor edit distances among 32 rollouts and reaches AUC 0.70. That score is absolute, though. An easy item converges to one rigid solution whether or not it was trained on, so difficulty and contamination are confounded. We reuse its diversity statistic as one input and remove the confound by measuring it relative to siblings.

The method comes from the data-mining use of maximum entropy as a background model. Maximum entropy models and subjective interestingness fits the maximum-entropy distribution matching known row and column sums of a binary database and flags structure that departs from it. Maximum Entropy Models for Iteratively Identifying Subjectively Interesting Structure in Real-Valued Data adds new constraints as knowledge accumulates. Our "known structure" is how a model behaves on the problem family plus the systematic shift that perturbing a problem causes. Contamination is the residual departure. The moment-based tail estimate follows Moment-Based Quantile Sketches and Reconstruction of Probability Density Functions from Channel Representations. Both show that a few moments under maximum entropy recover quantiles without spurious peaks or parametric distortion.

The closest entropy-based contamination work is different in kind. Entropy-Memorization Law correlates token entropy with memorization inside level sets and builds EMBEDI, a dataset-level test. EMBEDI needs roughly 1,500 sequences and leaves RL-trained models open, whereas we target single items in RL-trained models. Min-K%++ normalizes each token's log-probability by the vocabulary's mean and spread. That is a two-moment normalization across the vocabulary at one position, not a null model across problem variants, and it depends on the likelihood gap that RL erases. ABCD: All Biases Come Disguised shows that models prefer original MMLU-Pro option orders, without a formal test. Option permutation becomes one sibling type for us, but uniformity over permutations is the trivial maximum-entropy null, and we do not claim it as the contribution.

Research Program

For each item $x$ we build $K$ siblings. For math benchmarks (AIME, AMC, OlympiadBench, Minerva Math) we re-instantiate templates by changing constants and recompute answers with a symbolic solver. For GPQA Diamond we permute options and paraphrase with meaning verified by a second model and spot-checked by hand. For a budget of $N$ rollouts per prompt, we record a small statistic vector $s$: pass rate, Min-kNN structural similarity, mean rollout length and, when log-probabilities are available, mean answer-token log-probability.

The null for item $x$ is the maximum-entropy density on the bounded support of $s$ that matches the first $m$ sibling moments, each shifted by a perturbation offset $\delta_j$:

$$p_0(s) \propto \exp\Big(-\sum_{j=1}^{m} \lambda_j^\top \phi_j(s)\Big), \qquad \mathbb{E}_{p_0}[\phi_j(s)] = \hat{\mu}_j(\text{siblings}) + \delta_j .$$

Here $\phi_j$ is the $j$-th moment feature, $\lambda_j$ are Lagrange multipliers solved by convex optimization, and $\hat{\mu}_j$ is the sibling sample moment. The offset $\delta_j$ is the average original-minus-sibling difference measured on guaranteed-clean items, so that systematic difficulty changes from perturbation are absorbed into the background knowledge rather than flagged. We take clean items from post-cutoff questions in LiveBench and held-out uncontaminated items in the controlled setup. The contamination score is the tail probability $\Pr_{p_0}[T(s) \ge T(s_x)]$, where $T$ is the log-density ratio against the null in the direction of "better and more rigid". Item p-values are combined with Benjamini–Hochberg control for benchmark-level audits.

We also study which constraints matter. Setting $m=2$ on unbounded support reduces exactly to a Gaussian z-score. Higher $m$ on bounded support tests whether tail shape matters. Iteratively adding cross-statistic constraints, in the style of the iterative real-valued model, tests whether joint structure such as "high pass rate together with low diversity" carries signal beyond the marginals.

Baselines

The naive reference point is raw benchmark accuracy with no detector, which corresponds to calling items contaminated at chance, 50% AUROC. Next to it sits loss thresholding. The fragility paper reports that its AUROC drops about 16% after one epoch of clean GRPO, toward about 50%.

The strongest reported likelihood-based competitor is LiRA. It calibrates against a reference model and reaches 89.1% AUROC on SFT contamination of Qwen2.5-7B-Instruct before RL. It loses about 14% after one GRPO epoch, and on the post-RL DeepSeek-R1-Distill-Llama-8B setting it sits with the other detectors at 50–58% AUROC (all from On The Fragility of Benchmark Contamination Detection in Reasoning Models). We run LiRA with the same reference model and items as in that paper.

The strongest black-box competitor is Min-kNN distance at 32 rollouts, with AUC 0.70 reported across 1.5B–32B RLVR models in Detecting RLVR Training Data via Structural Convergence of Reasoning. For fairness, our method gets the same total generation budget of 32 rollouts per item, split between the original and its siblings. We sweep the split, and the tuning budget for $k$ and the split is identical across methods. Min-K% and Min-K%++ are included as log-probability baselines on the open-weight models. As a sanity check, Min-K%++ is also run on WikiMIA, where it reports gains of 6.2–10.5% AUROC over Min-K%, so we can confirm our implementation is faithful.

Two ablations isolate the ingredients. The first, the sibling-difference z-score ($m=2$, Gaussian), keeps the sibling design but drops the maximum-entropy tail model. The second, absolute statistics without siblings (our feature vector scored against a benchmark-wide null), keeps the maximum-entropy machinery but drops the siblings. Beating the baselines means two things in the post-RL settings: AUROC clearly above both 58% and 0.70, and an empirical false-positive rate at nominal $\alpha = 0.01$ that stays within a factor of two of nominal on clean items. The z-score ablation is expected to fail this second criterion.

Why not just…?

Why not just use Min-kNN or rollout entropy on the original item? Easy items collapse onto one solution path without any contamination, so absolute diversity measures difficulty as much as exposure. That confound plausibly caps Min-kNN near its reported 0.70. The no-siblings ablation measures how much of our gain comes from relative comparison. If that ablation matches the full method, the sibling design is unnecessary and we will say so.

Why not just compare accuracy on the original with accuracy on siblings, as EvoEval does with its 39.4% average drop across 51 models (Benchmark Data Contamination of Large Language Models: A Survey)? This is the real threat, because it captures most of the sibling idea. Our answer is narrow and testable. A raw drop or a Gaussian z-score on a pass rate near 1 with eight siblings produces badly calibrated tail probabilities, and auditing needs strict false-positive control rather than ranking. If the $m=2$ ablation matches the higher-moment null on both AUROC and calibration at $\alpha=0.01$, the maximum-entropy contribution reduces to a z-score. We would then report the result as a sibling-design paper.

Why not just calibrate with a reference model? LiRA is the reference-model approach, and it is precisely what the fragility paper shows collapsing after RL. For closed models, a reference trained on the unknown pre-training mixture is not available at all, as Detecting Pretraining Data from Large Language Models notes.

Why not just retire static benchmarks for LiveBench? Dynamic sets prevent future contamination but cannot audit the AIME or GPQA numbers already being cited. In our design they serve as the clean calibration source for the offset $\delta$.

Proposed Experiments

The main experiment replicates the controlled two-stage setup of the fragility paper, where ground truth is known. Stage I contaminates Qwen2.5-7B-Instruct by SFT on a random half of AIME, AMC, OlympiadBench, Minerva Math and GPQA Diamond items, which inflates pass@1 by 8.8% there, then applies clean GRPO for one and several epochs. Stage II contaminates DeepSeek-R1-Distill-Llama-8B, which yields an 11.7% boost there. We report item-level AUROC, true-positive rate at 1% false-positive rate, and empirical false-positive rate at nominal levels on held-out clean items, comparing every baseline across the GRPO epochs.

A second experiment separates generalization from memorization. In addition to direct item contamination, we contaminate with chain-of-thought data for other members of the same template family. Our hypothesis predicts low scores here because siblings are lifted too, whereas the fragility paper would label these items contaminated. We report both the AUROC and the fraction of measured pass@1 inflation located on flagged items.

The decisive checks are three. The first is whether the method stays above 0.70 AUROC after multiple GRPO epochs while LiRA and loss fall toward 50%. The second is whether the higher-moment null keeps the false-positive rate near nominal where the Gaussian ablation does not. The third is whether per-item scores correlate with per-item measured inflation, meaning the original's pass rate minus the clean-model pass rate. A null result on the first check, with siblings no better than absolute statistics, would falsify the core claim. A null result on the second would reduce the maximum-entropy component to a z-score. Finally, we apply the audit to public open-weight reasoning models on AIME and GPQA. There the output is a set of p-values with no ground truth, interpreted only alongside the controlled results.