When a coding agent cleans a dataset, fits a model, and reports a score, most of the ways it can cheat without meaning to are ordering mistakes. Each operation is individually correct. The agent normalizes, imputes, selects features, and splits the data. But one step runs too early: the scaler is fitted before the train/test split, or features are built from future time periods. Current agent benchmarks cannot see this. They check the final state of the sandbox or run a test suite, and by design they ignore the path the agent took. Leakage also raises the final score, so outcome-based evaluation rewards exactly the error it should catch. We propose to catch these errors without reading or understanding the agent's code. We replay the agent's recorded commands inside a frozen, reproducible container, once on the real data and once with the held-out data perturbed. Then we find the first step at which anything on the training side changes. If training-side artifacts react to test data before evaluation, the pipeline leaked, and the onset step says where.
The idea borrows its central move from a 1969 study of speech errors in aphasia. There, errors were explained purely by the positions of meaningless units such as phonemes, not by their meaning. It uses the hermetic containers now standard in agent evaluation as the thing that makes counterfactual replay possible. It targets the oldest problem in dataset transparency: pipelines whose written description does not match what was actually executed.
Core Claim
Our claim is that a large class of data leakage in agent-executed pipelines is a seriation error: correct units in the wrong order. It can be detected content-blind, by an order-sensitive invariance test run through deterministic replay. Let $s_t$ be the training-side state of the sandbox after step $t$ of a recorded trajectory. This means the persisted files and artifacts derived from training rows. Let $s'_t$ be the same state when the trajectory is replayed with the held-out partition perturbed while keeping its schema. The influence onset is
$$\tau = \min\{\, t : d(s_t, s'_t) > \epsilon \,\},$$
where $d$ is a file- or row-level divergence and $\epsilon$ is a noise floor estimated from two unperturbed replays. A pipeline is flagged as leaking if $\tau$ falls before the step that legitimately consumes the test partition for prediction. The same test with future time slices perturbed covers temporal leakage. The criterion checks a partial order (training state must not depend on test data) rather than a fixed action sequence. Many valid solution paths therefore pass. This reconciles the path-independence argument of state-diff benchmarks with the need to audit the trajectory.
Where This Sits
The sequencing lens comes from Analogy Between Sequential Errors in Aphasia and the Behavior of a Cybernetic System (SARF). That study showed that aphasic errors take three stereotyped forms, reproducible by stressing a machine buffer: perseveration (a unit repeats because it was never cleared), metathesis (two units swap), and anticipation (a later unit fires early). We use this as an operational labeling scheme rather than a claim about language-model cognition. Fitting on test data before the split is an anticipation. Swapping split and preprocessing is a metathesis. Reusing a stale fitted artifact from an earlier failed attempt is a perseveration. The authors themselves caution that the machine–human mapping is an analogy, and we keep it at that level. Machine Translation Robustness to Natural Asemantic Variation supplies a second, secondary ingredient. It measured whether translation models give consistent outputs under meaning-preserving rewordings, using a consistency metric called CONSIST. We borrow the idea of consistency under surface variation, but apply it to the process an agent executes rather than to output text.
The obvious combination would be to classify agent errors into perseveration, metathesis, and anticipation. We rejected it as the core contribution. Agent-Diff: Benchmarking LLM Agents on Enterprise API Tasks via Code Execution with State-Diff-Based Evaluation already catalogs recovery behaviors such as retry_same, which is perseveration under another name. A repeated-command counter would recover most of that taxonomy. A relabeling adds little. Detecting misordering whose effect is hidden inside data is different: no string heuristic sees it, and the stakes are well documented.
Two papers motivate the target. Leakage and the Reproducibility Crisis in ML-based Science found 329 papers across 17 fields affected by leakage. In the civil war case study, correcting temporal leakage and imputation shrank a complex model's AUC lead over logistic regression from 0.14 to 0.01. Reproducibility of deep learning in digital pathology whole slide image analysis showed that omitted preprocessing details alone drove reimplemented Camelyon models from 98.4% to 80.5% patch accuracy. Both papers answer with static reporting: Model Info Sheets and a 12-point checklist. We aim to generate the equivalent record from execution.
Several papers constrain the design. State-diff evaluation, as in Agent-Diff and AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents, deliberately discards order, so our check must be order-sensitive without being path-specific. Audit-Grade Harness for Agent-driven Scientific Computation Workflows pins containers, keeps an append-only log, and has a separate language-model verifier read the log. It showed that matching outcomes can hide fabrication and protocol deviation, but the file notes its verifier's confidence is uncalibrated. That is the gap a mechanical check could fill.
Other papers supply the method. Copy-on-Write Scoring: Application-Specific Agent Evaluations isolates agent writes into separate change tables and separates structural from content correctness. We reuse that snapshot-per-step idea at the filesystem level. A Qualitative Study on the Sources, Impacts, and Mitigation Strategies of Flaky Tests argues that hermetic environments remove run-to-run leakage, which is the precondition for a meaningful noise floor $\epsilon$.
The application regime is CoDA-Bench: Can Code Agents Handle Data-Intensive Tasks?. It drops agents into Linux sandboxes holding hundreds of Kaggle-derived files, and its best agent reaches 61.1% execution accuracy. It scores only the end result, so how much of that accuracy rests on leaky pipelines is unknown.
Research Program
We will build a replay harness around a pinned container. The harness records every shell command and script invocation the agent executes, together with copy-on-write snapshots of the filesystem after each step. It then re-executes the recorded command sequence, not the agent. We replay commands rather than re-query the agent because we are auditing the pipeline that actually ran, and because it avoids model sampling noise.
Each audit uses three replays. Two use the original data and establish $\epsilon$. One uses a perturbed held-out partition: values are resampled within columns so the schema and types are preserved, and labels are shuffled.
On tasks where we control the data, the harness injects a row-key column. This lets divergence be attributed to training rows exactly. On tasks we cannot modify, divergence is measured at file level on artifacts written before the first legitimate prediction step. That step is identified as the first command whose outputs are prediction or metric files.
The onset step, together with a few rule-based features, maps each flagged case to anticipation, metathesis, or perseveration. The rule-based features are whether the divergent artifact was written by a fit-like call, whether it predates the split, and whether it survives from an earlier attempt. The harness also emits a machine-generated provenance record. The record is organized around the fields of Fell et al.'s checklist, so the checklist is filled from what executed rather than from memory.
Known limitations must be tested rather than assumed away. Replay may diverge because of genuine nondeterminism, such as threading or unseeded randomness; the two-replay noise floor exists to catch this. Some leakage does not route through the held-out file at all, such as duplicate rows across files in a noisy sandbox. We will measure this blind spot explicitly rather than claim coverage of it. Replay also roughly triples execution cost. We will report that cost next to the 1.7% to 9.9% audit overhead SciAgent reports for its verifier.
Baselines
The naive reference point is outcome-only scoring: CoDA-Bench execution accuracy, or a state diff and test suite in the style of Agent-Diff. By construction it flags no leakage, and it rewards leaky runs whenever leakage improves the metric. We run it on identical trajectories. Its reported ceiling on this regime is 61.1% execution accuracy, and 49.6% on the hard subset, for Mini-SWE-agent with GPT-5.5 in CoDA-Bench. Beating it means showing that a measurable share of those successes is leaky and should be removed.
The strongest principled competitor is the sequestered hold-out check advocated by Kapoor and Narayanan's Common Task Framework. The test labels stay outside the container, and a run is flagged when the agent's self-reported score exceeds its score on the hidden labels. It is strong because it measures inflation directly and needs no instrumentation. We give it the same trajectories and calibrate its flag threshold on the same clean-pipeline set used to set $\epsilon$.
The strongest learned competitor is a fresh-context language-model verifier as in SciAgent. It reads the full append-only log and final scripts and is prompted to flag and localize leakage. We give it the same backbone budget as the agent, and tune the prompt on the same development split used to tune our divergence threshold. We found no reported result on Lacuna for the leakage-detection accuracy of any LLM trajectory verifier. The comparison is therefore new measurement rather than a match against a published number.
A static-analysis baseline pattern-matches the final script for known leaky idioms, such as fitting a transform on the full frame before train_test_split.
Two ablations isolate our key ingredients. The first compares only final states rather than per-step snapshots. It tests whether step-level onset matters for localization and for separating legitimate prediction divergence from leakage. The second drops the noise floor and uses $\epsilon = 0$. It tests whether reproducibility of the container, rather than perturbation alone, carries the method.
Beating the baselines means two things on the planted suite described below. First, higher recall at a matched false-positive rate of at most 5% on clean pipelines, with bootstrap confidence intervals that do not overlap the best baseline's. Second, correct localization of the offending step in a clear majority of flagged cases, which neither the sequestered check nor outcome scoring attempts.
Why not just…?
Why not just keep a hidden test set outside the container? This is the serious threat, and we treat it as the main comparison. It detects that inflation occurred, but only when the inflation is large enough to separate from sampling noise on a single task. Mild leakage, such as full-data normalization, may shift scores by little while still invalidating the claim, much like the 0.14-to-0.01 gap Kapoor and Narayanan uncovered only through careful re-analysis. It also cannot say which step leaked. And it requires a hidden label set, which is unavailable when auditing trajectories after the fact, for example when filtering generated training data. The planted-suite experiment stratifies leakage by effect size. If the hidden-set check matches our recall across all strata, we will say so and reposition the replay audit as a localization and training-data-filtering tool.
Why not just have a language model read the log, as SciAgent does? A verifier can often spot a textbook fit-before-split in one script. But the file reports that such verifiers have uncalibrated confidence and are prone to hallucination on long logs. In sandboxes like CoDA-Bench's roughly 980 files, leakage can travel through intermediate CSVs across several commands, where reading code is least reliable. Our experiments separate single-script from multi-step indirect leakage precisely to show where, if anywhere, the verifier holds up.
Why not just lint the final script? Agents' final scripts often differ from what actually ran: earlier attempts leave artifacts behind, and notebooks run out of order. The perseveration category, a stale fitted artifact reused after a retry, is invisible to any analysis of the final file. The static baseline and the perseveration stratum of the planted suite test this directly.
Proposed Experiments
The first experiment is a planted-leakage suite with ground truth. We take CoDA-Bench tasks that involve fitting and evaluating a model, using the released RUC-DataLab/CoDA-Bench data. We write clean reference pipelines, then create leaky variants following Kapoor and Narayanan's categories: preprocessing before the split, imputation on the full data, temporal leakage, and stale-artifact reuse. Each variant exists in a single-script form and in a multi-command form spread across intermediate files, and each is labeled with its true offending step. We report detection AUROC, recall at 5% false-positive rate, and localization accuracy against every baseline and ablation.
The second experiment applies the audit to real agent trajectories on the same tasks. We estimate how often agents leak, and how much of each agent's execution accuracy remains once flagged runs are removed. We hand-inspect a stratified sample to estimate the audit's precision in the wild.
The third, secondary experiment borrows the rewording methodology of Bremerman et al. We paraphrase each task instruction in meaning-preserving ways, such as reordering clauses or swapping "hold-out" for "validation", and ask whether the leakage rate changes. The consistency of an agent's process under surface rewording is the process-level analogue of CONSIST. It tests whether the reproducibility guarantee of a pinned container is broken at the instruction level, the concern raised by the 54% vocabulary-mismatch effect in the CoW Scoring study.
The decisive check is simple. If the sequestered hold-out check matches the replay audit's recall at equal false-positive rate on the multi-command and perseveration strata, then order-sensitive replay adds only localization, and the paper's claim narrows accordingly. If replay wins there, content-blind ordering checks become a practical transparency tool for agent-run science.
References
- Analogy Between Sequential Errors in Aphasia and the Behavior of a Cybernetic System (SARF)
- Machine Translation Robustness to Natural Asemantic Variation
- Agent-Diff: Benchmarking LLM Agents on Enterprise API Tasks via Code Execution with State-Diff-Based Evaluation
- Leakage and the Reproducibility Crisis in ML-based Science
- Reproducibility of deep learning in digital pathology whole slide image analysis
- AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents
- Audit-Grade Harness for Agent-driven Scientific Computation Workflows
- Copy-on-Write Scoring: Application-Specific Agent Evaluations
- A Qualitative Study on the Sources, Impacts, and Mitigation Strategies of Flaky Tests
- CoDA-Bench: Can Code Agents Handle Data-Intensive Tasks?