When a language model repairs code from error messages, it usually sees a traceback, edits, and tries again. Gains appear in the first two or three rounds and then stall. Our explanation for the stall is that the model never states what it expects its code to do, so a failure tells it only that something is wrong, not where its understanding broke. We propose a training setup in which the model must write down, before execution, the exact output its own program will produce on each test input. The sandbox then runs the code, and the model is rewarded for two things: passing the tests, and predicting its own program's behavior correctly. The gap between predicted and actual output becomes a precise, checkable signal of where the model's mental picture of its code is wrong. That signal is useful during training as a reward and during inference as a guide for where to spend more effort.
In plain terms, we turn the model's "internal world model," meaning its ability to mentally simulate what code will do, into something trained and scored against reality on every repair turn. We do not treat it as a black box absorbed during pretraining. The training method is group relative policy optimization (GRPO), a reinforcement learning method that scores each sampled answer against the other answers sampled for the same problem and needs no separate value network. The prediction reward has two useful properties. It is verifiable, because it is exact match against real execution. It also varies even when every sample in a group fails the tests, and those all-fail groups are exactly where standard GRPO gets zero gradient.
Core Claim
Our claim is that self-prediction accuracy can serve as a dense, unhackable-by-construction process reward for multi-turn code repair. We expect this reward to raise the point at which iterative refinement plateaus. The evidence would be higher pass rates after more repair turns at matched compute, compared with multi-turn GRPO trained on test outcomes alone. We further claim that the same predictions give a cheap signal at inference time for deciding which candidate programs to revise and when to stop.
The closest existing work combines pieces of this but never makes self-forecasting the trained object. MURPHY extends GRPO to multi-turn repair trees on real sandbox feedback. It propagates the best descendant's reward upward (its Max Reward Strategy, MARS) and reports up to an 8% absolute pass@1 gain over compute-matched single-turn GRPO on KodCode. Its reward, however, is purely the test outcome. Posterior-GRPO shows that outcome-only RL rewards accidental passes reached through bad reasoning. It fixes this with a learned reasoning-quality reward that only counts when the code passes. That gate zeroes out every failing turn, which is fatal in multi-turn repair, and the learned reward can itself be gamed.
CWM shows that mid-training on 120M Python execution traces yields over 96% exact match on trace prediction. That simulation ability is never tied to the model's own candidate programs inside an RL loop. AutoCoA trains a model to imagine tool responses and does part of its GRPO training inside that imagination. It works only on retrieval QA, and its imagined feedback replaces reality instead of being graded against it.
Our proposal differs from all four in the same way. The model's forecast is always paired with a real execution, and the agreement between the two is the reward.
Why this, and not the more obvious combination
The obvious idea at this intersection is to use an internal execution model to imagine sandbox feedback and prune rollout branches, which saves sandbox calls during multi-turn GRPO. We considered it and rejected it. For function-level benchmarks like KodCode and LiveCodeBench, running unit tests costs far less than generating the tokens. Replacing execution with simulation therefore saves little and adds hallucination risk. A skeptic would rightly say "just run the tests."
LiveCodeBench supplies the more interesting fact: models that generate code fluently often cannot predict that code's output. This gap between generating code and simulating it motivates our proposal. Training against it should help repair, because a model that cannot predict its program's output cannot reason about why that output is wrong.
Research Program
Each repair turn has a fixed format. The model sees the problem, a set of test inputs drawn from the benchmark's tests (not chosen by the model), and the history of earlier attempts. It emits three things: a program, a predicted output for each input (or a predicted exception type and line), and then a stop-or-continue decision. The sandbox executes the program. Feedback for the next turn includes the actual outputs and, for each input, whether the model's own forecast was right.
The per-turn reward is
$$R_t = \mathrm{MARS}(R^{\text{out}})_t + \lambda \, \frac{1}{|X|}\sum_{x \in X} \mathbb{1}\big[\hat{y}_t(x) = \mathrm{exec}(c_t, x)\big],$$
The symbols are as follows. $R^{\text{out}}$ is the binary test-pass reward, propagated up the repair tree with MARS as in MURPHY. $X$ is the set of provided inputs. $\hat{y}_t(x)$ is the model's forecast at turn $t$. $\mathrm{exec}(c_t, x)$ is the real output of that turn's program $c_t$. $\lambda$ is a weight that we keep small enough that the largest prediction reward is below one test pass.
The prediction term is deliberately not propagated across turns. It measures that turn's self-knowledge, not its contribution to later success. This is how we reconcile MURPHY's credit propagation with Posterior-GRPO's demand for process-level grounding.
Two predictable hacks need guarding. The first is writing code that crashes in an easily predicted way. Forecasts of exceptions only earn credit when the type and line both match, and we monitor crash rates across training. The second is writing trivially predictable code, which the dominant outcome term should discourage. We sweep $\lambda$ and report both hacks directly.
At inference time we use a "surprise-guided" search. We sample $K$ candidates with forecasts and execute all of them, since execution is cheap. A candidate whose forecast matched execution but fails tests has a correct model of a wrong program, so it receives a targeted edit. A candidate whose forecast mismatched has a localized misunderstanding, so the mismatched inputs are fed back as the primary error signal. The search stops when visible tests pass and forecasts match. Compute is measured in generated tokens plus sandbox calls.
We will train Qwen3 and OLMo models at 1.7B–7B on KodCode, matching MURPHY's setting so that numbers are comparable.
Baselines
The naive reference point is the base model, prompted for linear self-repair from tracebacks with the same turn budget. This yields the plateau curve that Yan et al. describe, with gains flattening after two or three iterations. Our method must bend that curve, showing gains that continue past turn three.
The strongest existing competitor is MURPHY: multi-turn GRPO with MARS on the same KodCode samples, the same model, the same number of rollouts, and the same turn depth. Its reported number is up to an 8% absolute pass@1 gain over compute-matched single-turn GRPO on KodCode (MURPHY). We will rerun it ourselves rather than compare to the reported figure, with an equal hyperparameter search budget for both methods. Beating it means higher pass@1 on held-out LiveCodeBench problems and on BigCodeBench-Hard at matched training compute. For BigCodeBench-Hard, there is no reported MURPHY number on Lacuna.
Single-turn GRPO and Posterior-GRPO are additional references. Posterior-GRPO reports a 4.5% gain over outcome-only RL on LiveCodeBench at 7B (Posterior-GRPO). We will run it with its gated thinking reward applied naively at each turn, which tests our claim that the gate starves failing turns.
The key ablation isolates the prediction reward. We train an otherwise identical model that still writes forecasts, and still sees them alongside actual outputs, but receives no reward for their accuracy. If this ablation matches our method, the benefit comes from the prompt format, not from training self-prediction.
For inference-time search, the baseline is best-of-N sampling with visible-test filtering at equal token-plus-execution budget. We also compare against CodePRM's generate-verify-refine loop, which reports APPS Competition Pass@1 rising from 16% to 29%. For self-repair, the LiveCodeBench paper reports GPT-4-Turbo improving from 24.5% to 36.9% with feedback; we cite this for context, not as a scale-matched target.
Why not just…?
Why not just run MURPHY longer or with more rollouts? Its signal is binary and outcome-only. When all rollouts for a hard problem fail, the group-normalized advantage is zero and the problem contributes nothing to learning. Our prediction term varies inside such groups. The decisive comparison is MURPHY given extra rollouts equal to our forecasting token overhead. If that version closes the gap, our idea is merely a compute reallocation, and we will report it as such.
Why not just mid-train on execution traces, as CWM does, and then run standard multi-turn GRPO? This is a real threat. CWM's over-96% trace-prediction accuracy suggests generic simulation skill may transfer to repair on its own. We test it directly. We add an auxiliary execution-prediction dataset, generic programs with traced outputs, to MURPHY training at the same token count as our forecasts. If generic simulation practice matches self-forecasting under RL, the coupling to the model's own code is not what matters.
Why not just use a separate critic to predict test outcomes? LLM critics reach 91.6% F1 on test outcomes on SWE-bench-Lite, but they need gold reference tests. CodePRM, likewise, needs a separately trained verifier. Neither changes the generator's own understanding of its code. Our ablations against CodePRM at matched compute will show whether internalizing the forecast beats outsourcing the judgment.
Why not just simulate execution to skip the sandbox? For these benchmarks the sandbox is cheap, and simulated feedback compounds errors. We therefore keep real execution on every turn and claim no savings in sandbox calls.
Proposed Experiments
Training uses 1,000 KodCode problems, as in MURPHY. Evaluation covers four benchmarks. On LiveCodeBench we measure Code Generation, Self-Repair, and the Code Execution scenario, the last of which directly measures whether self-forecasting transfers to general execution prediction. On BigCodeBench-Hard we measure library-heavy code. On APPS Competition we compare against CodePRM. We report pass@1 after each turn up to eight turns, alongside cumulative tokens and executions.
The first decisive check is the plateau curve. Our method should keep improving after turn three, while MURPHY and the unrewarded-forecast ablation should flatten.
The second is mechanistic. On held-out problems we test whether turns with a forecast mismatch precede successful repairs more often when the mismatched inputs are shown, compared with showing only a traceback. As an intervention, we strip forecasts at test time from the trained model. A large drop would confirm that the forecasts are actively used, not just a training regularizer.
The third tracks the share of all-fail groups that contribute nonzero gradient, along with crash-rate and trivial-code statistics across the $\lambda$ sweep. These numbers will tell us whether the dense signal helped or the model learned to game it.
References
- MURPHY: Multi-Turn GRPO for Self Correcting Code Generation
- Posterior-GRPO: Rewarding Reasoning Processes in Code Generation
- CWM: An Open-Weights LLM for Research on Code Generation with World Models
- Agent models: Internalizing Chain-of-Action Generation into Reasoning models
- LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code
- CodeTransOcean: A Comprehensive Multilingual Benchmark for Code Translation
- CodePRM: Execution Feedback-enhanced Process Reward Model for Code Generation
- Large Language Model Critics for Execution-Free Evaluation of Code Changes