When a language model reasons step by step, most of what it writes is not new. It restates a premise, carries a value forward from three steps ago, or repeats an assignment it already made. Only a few tokens per step are actually derived. Many failures in long reasoning are carry-forward failures rather than deduction failures: a quantity silently changes, an entity is swapped, a string loses a character. Program execution is the one setting where we know exactly which is which. A debugger log shows, for every value on every line, whether it was just computed or simply persisted from an earlier state. We propose to use that record to teach a model a copy discipline: when to point back and reproduce an earlier span verbatim, and when to generate something new. We then test whether that discipline, learned on code, carries over to natural-language logic puzzles and planning, where no such labels exist.
The mechanism is a pointer-generator: a decoder that, at each step, mixes ordinary next-token prediction with a "copy" distribution that points at earlier positions in the context. Pointer-generators have been used to keep summaries and answers faithful to a source document. Here the pointer targets the model's own reasoning trace as well as the prompt, and the gate that decides copy-versus-generate is trained on exact provenance labels extracted from execution. The claim is that part of what code teaches about reasoning is state binding, and that making binding an explicitly supervised, separate decision transfers better than hoping it emerges from next-token training on traces.
Core Claim
Execution-trace supervision already transfers to non-code reasoning. Chain of Execution Supervision Promotes General Reasoning in Large Language Models trains on narrated state changes (TracePile) and reports gains on Zebra-style puzzles and GraphWiz. Code Execution as Grounded Supervision for LLM Reasoning turns Snoop debugger logs into chain-of-thought and reports shorter, less circular reasoning. CodeI/O condenses code input-output prediction into rationales. All three treat every trace token the same way, as next-token prediction over the vocabulary. They discard the one piece of information that execution uniquely provides: for each value, its provenance. Provenance means whether the value was read unchanged from an earlier state or the prompt (and from which position), or freshly produced by an operation.
Our hypothesis has three parts. First, supervising a copy gate and pointer with provenance labels on code traces lowers carry-forward errors on code execution. Second, the learned gate generalizes to natural-language reasoning, so it points back when restating established facts and releases generation during deduction. Third, the gains should be concentrated on carry-forward errors rather than on computation errors. The third part is what makes the hypothesis falsifiable. If accuracy rises but the error breakdown does not shift toward fewer carry-forward mistakes, the mechanism story is wrong.
This speaks directly to a tension in the pointer-generator literature. Dr. Summarize found that pushing the model toward copying, via a penalty on the generator probability, was its most effective factuality intervention for clinical summaries. Get To The Point found its model stayed too extractive to beat a lead-3 baseline. Reasoning needs both behaviors in alternation: copy during setup and state carry, generate during inference steps. ZebraLogic makes clear that copying premises alone cannot resolve constraint conflicts. Neither paper had a signal telling the model when to copy. Execution provenance supplies that signal without anyone hand-labeling "reasoning phases."
Research Program
Base decoder. We start from a decoder-only model (Qwen3-4B and Qwen3-8B as primary models, plus a 1.5B Qwen model for the planning comparison). We add a pointer-generator head in the parameter-light style of A Copy-Augmented Generative Model for Open-Domain Question Answering. That work reused existing decoder cross-attention as the pointer and added only 1,537 parameters. In a decoder-only model we reuse self-attention from one middle layer, with a learned weighting over heads, as the pointer distribution $a^t$ over all previous positions (prompt and generated trace). A linear gate on the hidden state produces $p_{gen}$. The output distribution is the standard mixture
$$P(w_t) = p_{gen}^t\, P_{vocab}(w_t) + (1 - p_{gen}^t) \sum_{i < t:\, w_i = w_t} a_i^t,$$
where $w_i$ is the token at earlier position $i$. The added parameter count stays in the thousands, so any effect cannot be attributed to extra capacity.
Provenance labels. We re-run the PyEdu-R Snoop pipeline of Jung et al. Each value printed in a log line is tagged as carried (identical to a value in the previous state or in the input, recorded with the position where it was last written) or computed (the output of an operation on that line). Their training traces are natural-language translations by a larger model, not raw logs, so we align translated tokens back to log values by exact string match on literals and variable names. Tokens that match a carried value get label $y_t = 0$ plus a pointer target $j_t$, the earlier trace or prompt position holding that value. Matched computed values get $y_t = 1$. Unmatched connective prose is left unlabeled. We will report alignment coverage honestly; if it falls below roughly half of value tokens, that is itself a finding about how much translation erodes provenance.
Training objective. The loss adds two terms to ordinary language modeling, on labeled positions only:
$$\mathcal{L} = \mathcal{L}_{LM} + \lambda_g\, \mathrm{BCE}(p_{gen}^t, y_t) - \lambda_p \,[y_t = 0] \log a^t_{j_t}.$$
Here $\mathcal{L}_{LM}$ is the negative log-likelihood of the mixture above. The second term is binary cross-entropy teaching the gate the carried/computed label. The third term, applied only to carried tokens, teaches the pointer to attend to the value's defining position $j_t$. Multi-token values are handled as copy runs: once a pointer lands on the first sub-token of a span, the supervised target advances position by position. This targets the partial-copy failures that sub-word tokenization causes, which CRUXEval identifies as a major source of execution-simulation errors.
Analysis. For each evaluation trace we log $p_{gen}$ per token. We then ask whether the gate, never trained on natural language puzzles, drops when the model restates an established assignment and rises on derived ones. On ZebraLogic this is checkable against the solution grid. We also measure a self-inconsistency rate: the fraction of traces in which an attribute assignment stated earlier is later restated differently. This is the natural-language analogue of a carry-forward error. It is related to the drift patterns in Characterizing and Mitigating Reasoning Drift in Large Language Models, which describes looping and erratic jumps. Our metric isolates only the binding component of drift.
Prior work that constrains the design. Can Large Language Models Generalize Procedures Across Representations? found that plain supervised fine-tuning fails to transfer asynchronous-planning procedures from graphs to language because models latch onto surface keywords, and only a reinforcement-learning curriculum succeeded. Our method is supervised, so this is a direct threat. We test it head-on rather than assuming provenance supervision escapes it. The same paper reports residual entity-tracking errors even after reinforcement learning, which is exactly the error class we target.
Prior work that is consistent with the design. On Code-Induced Reasoning in LLMs shows that structural perturbations of code hurt transfer more than renaming variables. This fits our view that what matters is which earlier slot a value comes from, not its surface name.
Baselines
The naive reference point is the untouched base model with chain-of-thought prompting. It establishes how much any trace training helps at all.
The strongest existing method is execution-grounded fine-tuning as in Code Execution as Grounded Supervision for LLM Reasoning. It is the closest competitor because it uses the same raw data (Snoop logs over PyEdu-R) and already reports reduced repetition and circularity. The authors report Qwen3-8B average accuracy of 61.5% across MATH500, BBH, AGIEval, GPQA and LiveBench, against 57.8% for a CodeI/O distillation baseline. They also report 91.5% intermediate-step correctness on LiveBench versus 73.0% for CodeI/O. We will reproduce it on the same 15K instances, the same model sizes and the same number of optimizer steps and token budget as our method, with an equal hyperparameter search (same number of learning-rate and epoch configurations). Beating it means matching or exceeding its average accuracy while showing a statistically significant reduction in carry-forward errors on CRUXEval-O and in the self-inconsistency rate on ZebraLogic. An accuracy gain without the error-type shift would not count as support for the mechanism.
The ablation that isolates the key ingredient is an unsupervised pointer-generator. It is identical architecture and data with $\lambda_g = \lambda_p = 0$, so the gate is learned only through language modeling, as in See et al. and FiD-PGN. A second control uses shuffled provenance labels with the same label frequencies. If the unsupervised gate does as well, provenance is not the ingredient. If shuffled labels do as well, the auxiliary losses act as generic regularizers.
For the training-free alternative we run LargePiG decoding on top of the Snoop-fine-tuned model. LargePiG derives the copy probability from divergence between intermediate and final layers and was shown on query generation and QA with about 6% latency overhead on LLaMA2-7B. No reported result was found on Lacuna for LargePiG on multi-step reasoning or execution benchmarks.
For planning transfer we compare against the two-stage reinforcement-learning curriculum of Lin et al. Its 1.5B Qwen model reaches 0.782 accuracy on natural-language AsyncHow, matching zero-shot GPT-4o, and 0.573 on the AAVE variant.
For ZebraLogic we cite o1-full at 81.0% and DeepSeek-R1 at 78.7% overall (ZebraLogic) as a ceiling for context, not as a like-for-like comparison.
Why not just…?
Why not just fine-tune on execution traces and let attention learn copying on its own? This is the most serious alternative. Transformers copy well without help, and the Snoop-trained models of Jung et al. already show less repetition. The file contains no evidence that explicit copy supervision beats implicit learning in reasoning traces, so we do not assume it. The unsupervised-gate and shuffled-label ablations above settle it directly on matched data and compute. If provenance-supervised gating does not reduce carry-forward errors beyond the plain fine-tuned model, we will report the negative result. A negative result is still informative, since it would say that execution traces already teach binding implicitly.
Why not just use training-free copy decoding like LargePiG? It costs nothing to try, and we do try it. However, its copy signal tracks which tokens are "factual" versus "function words" via layer divergence. In a reasoning trace, a computed number and a carried number are both factual tokens, so that signal has no basis for telling them apart. That is precisely the distinction we need. The LargePiG-on-top comparison measures whether this concern holds in practice.
Why not just hand the problem to an interpreter or constraint solver? For ZebraLogic and CRUXEval a solver or Python interpreter would be exact, and we say so plainly. The target setting is reasoning without a formalizer, where the domain-confinement limitation noted for trace synthesis applies. That is why the transfer tests include AsyncHow natural-language planning and its dialect variant, where no compiler is available. The code benchmarks serve as the training domain and a diagnostic, not the end goal.
Why not just use self-consistency voting? Voting helps only if errors are independent across samples. Carry-forward errors caused by early lock-in, as characterized by Zhang et al., are exactly the kind that recur across samples. We include majority voting at 8 samples as a compute-matched-at-inference reference and report accuracy per generated token.
Proposed Experiments
The first experiment is in-domain. On CRUXEval-O and CRUXEval-I we compare all systems and hand-annotate 200 failures per system as carry-forward, computation, or other. For scale, CRUXEval reports GPT-4 with chain-of-thought at 81.9% (output) and 75.0% (input), and Code Llama 34B at 46% and 50%. The decisive prediction is that the provenance-supervised model cuts carry-forward failures substantially more than computation failures, relative to the Snoop fine-tuning baseline.
The second experiment tests transfer to natural-language logic. On ZebraLogic, stratified by difficulty tier, we report accuracy, self-inconsistency rate, and gate calibration. Gate calibration is the area under the curve for $p_{gen}$ separating restated from newly derived assignments, scored against the solution grid. Reasoning-model accuracy collapses at the X-Large tier (17% for o1-preview), so we do not expect a copy gate to rescue search-heavy puzzles. The prediction is gains on Small and Medium tiers, where binding errors rather than search dominate. This is itself a check against overclaiming. We also report the general suite used by Jung et al. to confirm there is no regression.
The third experiment tests transfer across representations. On AsyncHow natural language and NL-AAVE, at 1.5B scale, we compare four arms: provenance-supervised fine-tuning, plain fine-tuning, and the reinforcement-learning curriculum of Lin et al. with and without our gate added. If supervised provenance training alone approaches 0.782, it partly answers the question of whether supervised fine-tuning's transfer failure is a binding failure. If only the combination with reinforcement learning helps, the gate is a complement rather than a replacement.
The whole study is decided by one comparison: matched data, matched compute, and gate supervision on versus off. The outcome must be read off the error-type breakdown, not the headline accuracy.
References
- Chain of Execution Supervision Promotes General Reasoning in Large Language Models
- Code Execution as Grounded Supervision for LLM Reasoning
- CodeI/O: Condensing Reasoning Patterns via Code Input-Output Prediction
- Dr. Summarize: Global Summarization of Medical Dialogue by Exploiting Local Structures
- Get To The Point: Summarization with Pointer-Generator Networks
- ZebraLogic: On the Scaling Limits of LLMs for Logical Reasoning
- A Copy-Augmented Generative Model for Open-Domain Question Answering
- CRUXEval: A Benchmark for Code Reasoning, Understanding and Execution
- Characterizing and Mitigating Reasoning Drift in Large Language Models
- Can Large Language Models Generalize Procedures Across Representations?
- On Code-Induced Reasoning in LLMs
- LargePiG: Your Large Language Model is Secretly a Pointer Generator