Arbitrage Residuals as a Contamination Detector That Survives Reinforcement Learning

When a language model has seen a benchmark question during training, it can recall the answer without knowing anything about the question's neighbours. Suppose a model memorized that a problem's answer is 47. Ask it "is the answer above 40?", "above 50?", "below 45?" and it has nothing memorized to fall back on. It has to produce these answers some other way, and they will not line up with the 47 it recalled. Finance has a precise language for this kind of mismatch. A set of prices or probabilities is arbitrage-free when no combination of bets on it guarantees a profit. For threshold questions that means the probability of "answer above $t$" must fall as $t$ rises, the way an option price must fall as its strike rises. This note proposes measuring the size of one specific arbitrage: the gap between a model's direct answer and the arbitrage-free belief implied by its answers to logically related questions. That gap is used as a black-box, outcome-free test for benchmark contamination.

The timing matters because the standard contamination detectors have just been shown to fail. They look for unusually high confidence or low loss on seen items. On The Fragility of Benchmark Contamination Detection in Reasoning Models shows that supervised fine-tuning followed by reinforcement learning with clipped updates erases that confidence gap. In their experiments LiRA falls from 89.1% AUROC to near 50%, and ten standard detectors average 50–58% AUROC once models are tuned on chain-of-thought solutions. Reinforcement learning hides the memorized item's confidence. We conjecture it does not make the item consistent with logical siblings that were never trained on.

Core Claim

Contamination leaves a relational signature: the memorized "anchor" item is an outlier relative to the arbitrage-free fit of its unmemorized siblings. A detector built on this signature can work without log-probabilities, without ground-truth labels for the siblings, and without a pool of clean reference models.

The anchor-relative design is essential. Clean models are already very incoherent: Outcome-Free Arbitrage Audits and Coherence Repairs for LLM Forecasters finds 68.8% of complementary pairs incoherent across 15 models. It also finds that coherence barely tracks accuracy (Spearman $\rho=+0.15$). Separately, Rethinking LLM Confidence: From Calibration to Coherence reports that standard fine-tuning raises exploitability on paraphrased prompts by 171%.

These findings constrain the design. A raw "incoherence rate" would flag every post-trained model and could not tell contamination from ordinary sloppiness. What should distinguish contamination is where the incoherence sits. In a clean model, the direct answer and the sibling answers are noisy readouts of one belief. In a contaminated model, the direct answer is a separate memorized channel.

For item $i$, let $a_i$ be the model's direct answer and $q_i$ the vector of its answers to sibling questions. Let $\Pi(q_i)$ be the arbitrage-free belief obtained by projecting $q_i$ onto the coherent set and reading off the value it implies for the anchor question. The anchor residual is

$$r_i = \frac{|a_i - \Pi(q_i)|}{\tilde\sigma_i},$$

where $\tilde\sigma_i$ is the median leave-one-out residual among the siblings themselves. The siblings form a leave-anchor-out reference. Dividing by $\tilde\sigma_i$ normalizes away the model's baseline sloppiness on that item. Contaminated items should have large $r_i$; clean items should look like any of their siblings.

Research Program

The first component is a sibling generator that turns any item into a constraint family whose coherence can be checked without knowing the true answer.

Numeric-answer items (GSM8k, quantity-answer factual questions) become threshold ladders: "Is the answer greater than $t_j$?" for 10–15 values of $t_j$ spread around plausible values. The arbitrage-free condition is that these probabilities form a non-increasing survival function. The projection $\Pi$ is then isotonic regression, and the implied anchor value is the median of the fitted distribution.

This borrows the logic of the option-surface literature. Mixture of Normalizing Flows for European Option Pricing gets monotonicity and convexity in strike for free by fitting a probability density and integrating it. Incorporating Prior Financial Domain Knowledge into Neural Networks for Implied Volatility Surface Prediction enforces butterfly and monotonicity constraints with penalties. Both treat a violation as something to remove. We treat the size of the violation, located at a particular point, as the measurement. Our "density" is the model's implied belief about the answer rather than a market's risk-neutral distribution.

Binary forecasting items use the four constraint families formalized by Li & Sreedhar: complements, nested-threshold chains, Fréchet conjunction bounds, and entailments. CoherenceBench v0 already supplies 19 chains, 39 conjunction triples and 21 entailments. Li & Sreedhar provide only a closed-form repair for complementary pairs and leave joint repair across all constraint families unimplemented. We implement it, with Dykstra's cyclic projection as $\Pi$. That piece of engineering is needed for the detector.

The second component is elicitation. Probabilities come either from verbalized confidence or from answer frequencies over $n$ samples. The method therefore runs on APIs that hide token probabilities, which is a stated weakness of Min-K%++ and PaCoST.

The third component is a decision threshold. The null distribution of $r_i$ comes from the same model on items written after its training cutoff, in the spirit of the temporal splits behind WikiMIA and PaCoST's use of WMDP. This avoids the reference-model drift that weakens ConStat when most public models are themselves contaminated.

The fourth component is a controlled contamination testbed that follows the Tang et al. pipeline. We take Qwen2.5-7B and a Llama-family 8B model, both open-weight. We contaminate them by supervised fine-tuning on half of each benchmark's items, in three variants: verbatim answers, chain-of-thought solutions, and paraphrased items. We then run one epoch of clean GRPO, the clipped reinforcement-learning objective that hides contamination. The benchmarks are GSM8k, CoherenceBench v0 and WMDP. Membership labels are known by construction.

Baselines

Naive reference. Raw incoherence rate per item, meaning total arbitrage across the constraint family with no anchor normalization. We expect it to be dominated by the model-wide inflation of incoherence after reinforcement learning that Matta et al. document. If it matches our detector, the anchor-relative design is unnecessary.

Log-probability detectors. Min-K%++, which z-normalizes token log-probabilities and averages the lowest $k\%$, and LiRA, run on the open-weight models where log-probabilities are available. These are the strongest white-box competitors. Tang et al. report 89.1% AUROC for LiRA before reinforcement learning, falling to roughly 50% after it, and 50–58% for standard detectors after chain-of-thought tuning. For PaCoST, which runs a paired t-test on $P(\text{True})$ for original versus rephrased questions, no reported AUROC was found on Lacuna; we run it ourselves.

Strongest existing method: ConStat. It flags a model when its performance gap between original and rephrased benchmarks exceeds what a difficulty correction fitted across reference models predicts. It reports a 98.4% detection rate on controlled sample-specific contamination. Note that this is a detection rate, not an item-level AUROC, and it was not measured after reinforcement learning. We run ConStat with the same reference pool for both item-level and model-level decisions.

Fairness and the bar to beat. Every method gets the same contaminated checkpoints and an equal query budget per item, matched in model calls rather than prompts. Each also gets the same post-cutoff calibration set. Beating the baselines means post-reinforcement-learning item-level AUROC of at least 0.75 where the log-probability detectors sit near 0.5. It also means matching ConStat's model-level detection without using reference models.

Ablation 1: sibling-accuracy gap. The same siblings are scored by correctness against ground truth instead of by coherence. This isolates whether the arbitrage projection adds anything beyond "the model gets neighbours wrong."

Ablation 2: paraphrase siblings. Logical siblings are replaced with paraphrases. This isolates the claim that logical transformations, not mere rewording, are what reinforcement learning fails to cover up.

Why not just…?

Why not just use perplexity or n-gram overlap? N-gram overlap misses paraphrased contamination, as ConStat argues. Loss-based scores are exactly what Tang et al. show collapsing to chance after clipped reinforcement learning. We include both and expect them to fail in the post-reinforcement-learning condition. If they do not, the motivation for this work weakens and we will say so.

Why not just use ConStat's generalization gap? This is the real threat. Our detector is also, at heart, a test of whether performance generalizes beyond the memorized form. ConStat's advantages are reported strength and simplicity. Its disadvantages are that it needs ground-truth labels for the variants and a clean reference pool, and that it works at the level of a whole benchmark rather than a single item. Ablation 1 settles the question. If the sibling-accuracy gap matches the arbitrage residual, the contribution reduces to "outcome-free and reference-free." That still matters for forecasting questions that have not yet resolved, but it is a smaller claim, and we would present it as that.

Why not just test paraphrase invariance, as PaCoST does? Tang et al. find that chain-of-thought contamination teaches patterns that transfer to unseen instances. Rewording is the transfer most likely to survive. Asking for the complement of a threshold, or for a conjunction with an unrelated fresh event, requires a coherent belief rather than a recalled string. Ablation 2 tests this directly.

Why not just evaluate on fresh items (LiveBench, DyVal)? LiveBench and DyVal avoid contamination going forward. DyVal shows phi-1.5 and WizardMath-13B scoring near zero on generated tasks. Neither can audit which items of an existing static benchmark a given model has seen. We use fresh items only as the null calibration set, so the two approaches are complementary rather than competing.

Proposed Experiments

The decisive check is the 2×2 grid of {contaminated, clean} × {before, after GRPO} on GSM8k ladders and CoherenceBench v0 families. The claim holds only if AUROC for $r_i$ stays roughly flat across the reinforcement-learning step while LiRA and Min-K%++ fall toward 0.5. The claim fails if GRPO also restores sibling consistency. That outcome would mean reinforcement learning hides relational traces too, which is itself worth reporting.

The hardest case is the chain-of-thought contamination variant. With reasoning enabled, a model could re-derive the answer inside each threshold prompt. We therefore report results with reasoning on and with reasoning capped at a short budget. We expect the signal to shrink with long reasoning, and we will measure how much.

Model-level audit. We apply the calibrated detector to public checkpoints on GSM8k and ARC. We then compare the items it flags, and the share of each benchmark flagged, with ConStat's reported inflation of about 8.3% for Mistral-7B on GSM8k and about 6.6% for Llama-3-70B on ARC. The audit counts as successful if the two methods produce overlapping orderings of models even though they share no reference models.

Cost. We report model calls per audited item and how AUROC varies with ladder length $m$. This determines whether the residual is cheap enough to compute routinely for every benchmark item.