Leave-Out Audits: Measuring Benchmark Leakage in Cheap Domain Adaptation

Cheap ways of specializing a language model to a field like biomedicine, law or science now dominate practice. These include small low-rank weight updates (LoRA), searchable external document stores consulted at answer time, and patched vocabularies. Each depends on a domain corpus that nobody checks against the benchmark used to score it. When the scientific datastore is built from the same paper archive that the test questions were drawn from, a retrieval model can look up the answer's source, and the reported "domain adaptation gain" may partly be leakage. Existing contamination detectors are poorly suited to this case. They were designed to spot memorized pretraining text, and they are known to miss contamination introduced by fine-tuning.

We propose to use the low cost of these methods as the audit tool. Rebuilding a LoRA adapter, an n-token vocabulary, or a retrieval index takes hours on one GPU, not thousands of GPU hours. That makes the most direct contamination test affordable: remove every corpus document that resembles the test set, rebuild the adaptation, and see how much of the gain survives. A matched control removes the same amount of random domain text, which separates "lost leaked content" from "lost domain data." The result is a contamination estimate that is causal rather than inferred from likelihood quirks. Any third party with the corpus and a single GPU can rerun it. The contribution is this audit protocol, its calibration against planted leaks, and its application to published lightweight-adaptation results.

Core Claim

For any adaptation method whose rebuild cost is small, the share of a benchmark gain attributable to test-set leakage can be measured directly. The method is counterfactual removal with a matched random-removal control. This estimate is more reliable than detectors that read signals off the final model. Concretely, write $S(C)$ for the benchmark score after adapting on corpus $C$, $N_\tau$ for corpus documents within similarity threshold $\tau$ of any test item, and $R$ for a random subset of $C$ with the same token count as $N_\tau$. The leakage-attributable gain is

$$\Delta_\tau = \big[S(C) - S(C \setminus N_\tau)\big] - \big[S(C) - S(C \setminus R)\big] = S(C\setminus R) - S(C \setminus N_\tau).$$

A positive $\Delta_\tau$ that grows as $\tau$ loosens, from exact n-gram overlap through paraphrase-level embedding similarity to shared source document, is evidence that the gain depends on test-adjacent content specifically, not on domain text in general.

How this relates to prior work

The motivation comes from three findings. Towards Data Contamination Detection for Modern Large Language Models compared five detectors on frontier models and on a LLaMA-2-70B oracle fine-tuned on the full MMLU test set. In that oracle, Min-K% probability and canonical-order tests produced no clear signal, and the five detectors barely agreed with one another. Our setting is exactly fine-tuning-style contamination, so we do not rely on those detectors. ConStat defines contamination functionally, as a performance gap between original and rephrased benchmarks relative to clean reference models. It found inflation such as $\delta \approx 8.3\%$ for Mistral-7b on GSM8k. ConStat is the closest competitor. It differs from our protocol in two ways: it needs trusted reference models and rephrased reference benchmarks, which rarely exist for domain tasks like ACL-ARC or CBLUE, and it observes the final model instead of intervening on the training input. P⁴Bench prevents contamination by hiding answer keys behind zero-knowledge proofs. It works only for problems whose answers can be checked by a formula, such as satisfiability, so it does not cover the classification and extraction tasks used in domain adaptation. We see it as complementary.

The adaptation methods being audited supply the experimental objects. LoRA trains low-rank weight updates; on GPT-3 175B it cut trainable parameters 10,000x. AdaptiVocab adds multi-word domain tokens and tunes only the embeddings and the outer layers. It is evaluated partly on synthetic QA generated from the adaptation corpus itself, which is a built-in leakage risk. Retrieval-Augmented Domain Adaptation (RADA) retrieves passages from domain datastores, including S2ORC for SCIERC and ACL-ARC, and reports an average +1.59 points over standard fine-tuning. Because RADA can swap datastores without retraining, removal costs only a re-index, which makes it the cheapest method to audit. Nearest Neighbor Machine Translation shares that property. M2D2 supplies fine-grained domain corpora. Its finding that vocabulary overlap strongly predicts transfer (Pearson r ≈ −0.8) constrains our design: removing test-adjacent documents also removes in-domain vocabulary, so the random-removal control must be drawn from the same subdomain, not the whole corpus.

On reproducibility, the protocol yields a published, re-runnable number. Auditors release hashes of the removal sets and the rebuild recipe. A holder of a private corpus, as in Domain Adaptation for Enterprise Email Search or MIMIC-style clinical data, can run the same procedure in-house and report $\Delta_\tau$ next to the headline score. We do not claim to make such private reports cryptographically verifiable.

Research Program

The first component is the neighbor definition, at three levels of strictness: 13-gram overlap with test inputs or labels; embedding similarity above a threshold swept over several values; and provenance, meaning the corpus contains the source document the test item was extracted from. Provenance matters most for SCIERC and ACL-ARC, whose instances come from scientific papers that S2ORC plausibly contains.

The second component is calibration with planted leaks. We inject known test items into otherwise clean corpora at controlled rates (0.1%, 1%, 10% of test items) and in controlled forms (verbatim, paraphrased, answer-augmented with a worked rationale). We then check whether $\Delta_\tau$ recovers the planted effect for each adaptation method. This answers whether the audit works before it is pointed at published claims.

The third component is the audit of published results: RADA on SCIERC, ACL-ARC, CAIL2019-SCM and CBLUE, LoRA trained on the same domain corpora, and AdaptiVocab on M2D2 Earth Science, History of Physics, and Games & Toys. We expect retrieval to be most exposed, because lookup turns a single leaked passage directly into an answer. We expect vocabulary patching to be least exposed, because n-tokens encode surface frequency more than item content. We report a cost ledger for every audit: GPU-hours per rebuild, number of rebuilds, and seeds. This makes clear that the same protocol on full domain-adaptive pretraining would be prohibitively expensive.

Baselines

The naive reference is the status quo: the adapted model's reported score with no audit, alongside the unadapted base model. RADA's reported +1.59 average over fine-tuning, from RADA, is the gain whose leakage share we estimate. For AdaptiVocab end-task accuracy, no reported result was found on Lacuna. For the planted-leak study, the reference is the known injected effect. Any audit that recovers less than half of a planted 1%-rate paraphrased leak counts as failing.

The strongest existing detector is ConStat (ConStat). It reported detecting 98.4% of sample-specific contamination in its synthetic setup, against 71% for the best prior baseline. We run it on the same adapted checkpoints. For reference models, we use the same base model adapted with the same method on the leak-free corpus at several seeds, which gives ConStat a better reference set than it usually has. For rephrased benchmarks, we generate paraphrased test variants with an equal generation budget. Our audit beats ConStat if it has higher detection rate at equal false-positive rate on planted leaks, especially paraphrased and answer-augmented ones. A second requirement is that it does not flag clean low-rank adapters that are merely brittle to rephrasing; this is the confound ConStat cannot rule out.

The likelihood and overlap detectors, Min-K% probability and token-completion overlap as implemented in Towards Data Contamination Detection, are run on the same checkpoints. They are the cheapest competitors. We expect them to fail on fine-tuning-style leaks, consistent with that paper's MMLU oracle result.

Decontaminate-then-report is the practical alternative: filter $N_\tau$ once, rebuild once, and report $S(C\setminus N_\tau)$. It uses the same rebuild budget at a single threshold.

The key ablation drops the matched random-removal control and reports only $S(C) - S(C\setminus N_\tau)$. This isolates whether the control is necessary to avoid confusing domain-data loss with leakage. All methods share the same base models, corpora, seeds, and a fixed GPU-hour budget per condition.

Why not just…?

Why not just n-gram-filter the corpus before adapting? That is the decontaminate-then-report baseline, and for verbatim leaks it may be enough. It fails in two ways we can measure. Exact filters miss paraphrased and rationale-augmented leaks, which the planted study tests directly. A single filtered score also cannot tell you how much leakage remains or how much signal the filter removed. If an embedding-similarity filter at one threshold fully recovers planted paraphrased effects, the threshold sweep adds little. In that case we would report the simpler filter as sufficient and keep only the matched control and the provenance level.

Why not just run ConStat on the adapted model? ConStat is the real threat, and its 98.4% synthetic detection rate is strong. In domain settings, however, its trusted references and rephrased benchmarks must be constructed, and its signal is a rephrasing gap that low-rank or vocabulary-patched models may show for capacity reasons alone. The planted-leak comparison settles which audit is more accurate. Clean-adapter false-positive rates settle the brittleness question.

Why not just evaluate on freshly written test items? Fresh post-cutoff items avoid leakage, but domain benchmarks like CAIL2019-SCM or CBLUE are expensive to rewrite. Fresh items also do not explain published gains already on record, and they cannot be shared when the domain data is private. Where we can cheaply write a small fresh SCIERC-style set, we use it as an external check that $\Delta_\tau$ predicts the fresh-set gap.

Why not use full domain-adaptive pretraining, where retrieval shortcuts don't exist? Weights memorize too. Full pretraining's thousand-GPU-hour cost is exactly what makes leave-out audits infeasible for it. The protocol depends on rebuilds being cheap.

Decisive Checks

The protocol succeeds if two things hold. First, on planted leaks, $\Delta_\tau$ tracks the injected rate across verbatim, paraphrased, and augmented forms, and it stays near zero for clean adapters, while Min-K% and overlap do not. Second, on the published RADA suites, the audit returns stable estimates across seeds with confidence intervals narrower than the 1.59-point gain under study. Either outcome on the real benchmarks is informative. A large $\Delta_\tau$ on ACL-ARC or SCIERC with an S2ORC datastore would show that part of a celebrated efficiency gain is lookup of the test item's source. A near-zero $\Delta_\tau$, together with a validated calibration, would give the first causal evidence that those gains are genuine.