Learning to Repair Code From a Model's Own Diverse Failures

Language models are poor at fixing their own mistakes. Asked to review its own answer with no outside check, a model often argues itself out of a correct solution. Trained on artificially inserted errors, it learns to fix those errors but not the ones it actually makes. Code is a natural place to test a different recipe, because a program can be run and the interpreter returns an honest error message. We propose building repair training data from the model's own failures: sample many programs from the model with randomness turned up, run them, keep the ones that fail together with the real error messages, and pair each with a working fix. The central claim is that how diverse the failures are, controlled by the sampling temperature used to produce the broken drafts, decides whether the learned repair skill carries over to the model's real mistakes at test time. Using the model's own outputs matters, but it is not enough on its own.

This joins two findings that have not been tested together. In machine translation, training data generated by random sampling beats data from the single most likely output, because the most likely output collapses onto a narrow, "boring" slice of the distribution (Beyond the Mode; Understanding Back-Translation at Scale). In self-correction, fine-tuning on injected errors reaches 99% correction on those errors but only 8% error recognition on the model's own Sudoku mistakes, and models often announce "I made a mistake" and then repeat it (Synthetic Error Injection Fails to Elicit Self-Correction). Our hypothesis is that the second failure is a coverage problem of the same kind as the first. If so, it should be fixable with a sampling knob and real execution feedback, without reinforcement learning.

Core Claim

Supervised fine-tuning on (failing draft, real traceback, fix) triples improves repair of the model's natural failures on unseen problems, and the improvement depends on draft diversity rather than on draft count alone. At a fixed number of training pairs, drafts sampled across a sweep of temperatures should beat drafts taken from greedy decoding, which always picks the single most likely token. Drafts sampled across temperatures should also beat rule-based mutations of correct code. We expect the benefit to rise and then fall with temperature: very high temperatures produce broken code unlike anything the model writes at test time. Following the diminishing-returns result in Beyond the Mode, we also expect the gap to shrink as the number of distinct training problems grows. The practical claim is narrower and riskier. At matched inference compute, a model trained this way should beat simply drawing more samples and keeping one that passes the visible tests.

Relation to Prior Work

Three papers motivate the gap. Large Language Models Cannot Self-Correct Reasoning Yet shows that self-review without outside feedback lowers accuracy (GPT-4 on GSM8K falls from 95.5% to 89.0%). It also leaves open whether external signals such as execution change this. Our setting supplies exactly that signal. The error-injection study ( Synthetic Error Injection Fails to Elicit Self-Correction In Language Models) concludes that supervised methods fail because of distribution mismatch. However, it tested only rule-based perturbations on toy arithmetic and Sudoku, never errors sampled from the model itself or real interpreter output. Our design is a direct test of whether its conclusion holds once mismatch and coverage are separated. Fine-Tuning with Divergent Chains of Thought partly contradicts it: supervised training on sequences of attempts does induce self-correction when the earlier attempt sits in context. That work relied on prompt triggers rather than controlled sampling, and it used natural-language reasoning rather than code.

Two lines constrain the design. Inference-time debugging loops stall. Compiler feedback adds about 3 points and then plateaus in CodeTransOcean, and 64 rounds of Lean feedback reach only about 20% in VERINA. We therefore measure per-round gains, not just final pass rates. Evaluating Intermediate Reasoning of Code-Assisted LLMs shows that code which passes can still rest on brute force or ungrounded logic. A repair model trained only on test outcomes could learn to patch toward such shortcuts, so we reuse that paper's syntax-tree-based structure classifier as a check.

The closest method is TGPR. It uses reinforcement learning (GRPO) over refinement trees explored with Thompson sampling, a randomized way of choosing which branch of the tree to try next. Its reward requires a reference solution through CodeBLEU, a text-similarity score against that reference. We differ in three ways. We use plain supervised learning. We need no reference solution: any passing sample, including one from the same model, serves as the fix. Most importantly, we treat the diversity of the failure distribution as the controlled variable, whereas TGPR folds exploration into a search procedure without isolating it.

Research Program

The base model is a 7B open code model of the class used by TGPR, and training problems come from APPS, as in TGPR. For each training problem we draw drafts at temperatures from 0 (greedy) to 1.2, execute them in a sandbox, and record the failure: syntax error, runtime exception, wrong answer on a test, or timeout. The recorded evidence is the traceback or the failing input and output. The fix for each failure is a passing program for the same problem. To keep fixes close to the draft, we prefer the passing sample from the same model with the smallest edit distance to it. If none exists, we ask the same model to repair the draft until it passes. We also train a multi-round variant in which the first repair attempt fails and its own traceback feeds a second round, which directly targets the plateau.

The key experiment holds the number of training triples fixed and varies only where the broken drafts come from. There are five data arms. The first uses greedy on-policy drafts, meaning the model's own outputs at temperature 0. The second uses on-policy drafts sampled across the temperature sweep. The third uses a single high temperature. The fourth uses rule-based mutations of correct programs, the code analogue of error-injection training: flipped comparisons, off-by-one indices, dropped branches. The fifth uses failures sampled from a different, stronger model, which are natural errors but come from another model. Comparing the first and second arms isolates diversity. Comparing the fourth arm against the rest isolates natural versus artificial errors. Comparing the fifth arm against the second isolates whether the errors must be the model's own. For every arm we record how widely the training failures spread across error types, using the entropy of the failure-category histogram. We then check whether this spread predicts transfer. We also sweep the number of distinct training problems (1k, 5k, full APPS) to test the predicted narrowing of the diversity gap.

Baselines

The naive reference point is the untrained base model run in the same interpreter-feedback loop: it receives its failing program and the traceback and is asked to fix it, with the same number of rounds. This is the setting in which CodeTransOcean sees early plateaus. Any trained arm must beat it on repair success over the model's natural failures on held-out problems.

The strongest simple competitor is sample-and-filter: draw independent programs and return the first that passes the visible tests. Following Evaluating Large Language Models Trained on Code, repeated sampling with test filtering scales well, and Large Language Models Cannot Self-Correct Reasoning Yet finds that majority voting beats debate at equal cost (88.2% vs 83.0% on GSM8K with nine responses). We match it on generated tokens and on the number of executions: a draft plus $r$ repair rounds is compared against $r+1$ independent samples filtered by the same visible tests. Temperature for sample-and-filter is tuned on the same validation split with the same budget. Beating it means a higher hidden-test pass rate at equal tokens, across a range of budgets rather than at a single point.

The strongest existing trained method is TGPR, which reports greedy pass rates of 41.2 on LiveCodeBench (Oct 2024–Feb 2025) and 77.6 on MBPP for Qwen-7B, against 36.7 and 74.8 for GPT-4o-mini (TGPR). The file lists no released TGPR code. We will therefore reimplement plain GRPO with a test-pass reward on the same APPS problems and the same total rollouts as our data generation, and cite TGPR's numbers as an external reference only. Because TGPR uses reference solutions that our method does not, we also run our method with reference solutions admitted as fixes to show what that extra information is worth. Matching GRPO at equal sampling compute without reinforcement learning would count as a win. Beating it on the rate at which the model repeats the same bug would count even if pass rates tie.

For LiveCodeBench's self-repair scenario, the reported reference point is GPT-4-Turbo rising from 24.5% to 36.9% when given interpreter feedback (LiveCodeBench).

The key ablations are the data arms above. They separate diversity, on-policy origin, and natural versus artificial errors at a fixed pair count. They also include a version with tracebacks removed from the input, to measure how much of the gain depends on real execution feedback.

Why not just…?

Why not just sample more and keep what passes the tests? This is the real threat. On short, self-contained problems it may win outright, and we will report that honestly. The compute-matched curves decide it. Repair has a structural advantage when the first draft is nearly right, since a one-line fix is cheaper than a fresh correct sample. That situation should dominate on harder problems, where pass@1 is low but failures are partial. We therefore stratify results by problem difficulty. If sample-and-filter matches repair at every budget and difficulty level, the practical claim fails, and what remains is the scientific finding about why error-injection training does not transfer.

Why not just use reinforcement learning, as TGPR does? Reinforcement learning may well reach similar pass rates. Our question, however, is what property of the training failures makes repair transfer, and supervised arms with controlled data allow that attribution where reinforcement learning rollouts do not. The GRPO baseline at matched rollouts tells us whether supervised learning on well-chosen failures closes the gap. Our method also needs no reference solution. TGPR also reports slightly more hallucinated code structures under aggressive exploration, and we measure this in both methods.

Why not just inject rule-based bugs into correct code, which is cheaper? Synthetic Error Injection Fails to Elicit Self-Correction shows a collapse from 94% to 8% recognition when models trained this way meet their own errors. It is possible, though, that code mutations resemble real bugs more closely than digit swaps resemble arithmetic slips. The mutation arm tests this directly at matched pair count. If it matches the temperature-swept arm, our diversity story is wrong for code, and we would say so.

Why not just run the debugging loop longer at inference time? The file's evidence says gains stall after two or three rounds in CodeTransOcean, and near 20% after 64 rounds in VERINA. Our per-round curves show whether training moves the plateau or only shifts the first round.

Proposed Experiments

We train on APPS and evaluate on held-out sets: LiveCodeBench code generation and self-repair (contamination-free by date), MBPP and HumanEval, the Code Debug split of CodeEditorBench (about 20% pass across models reported in CodeEditorBench), and the Python-target setting of CodeTransOcean via its debugging success rate after $K$ rounds (DSR@K), reported there as 48.57% at DSR@0 and 52.57% at DSR@3 for GPT-3.5.

The primary metric is repair success on the model's own natural failures on held-out problems, broken down by failure type. We also report four secondary metrics. The first is the parrot rate: the share of repairs whose resulting program fails the same tests with a near-identical syntax tree. The second is the correct-to-incorrect flip rate when a passing program is submitted with a request to check it, the failure identified by Large Language Models Cannot Self-Correct Reasoning Yet. The third is pass rate per repair round up to five rounds. The fourth covers program-of-thought math problems (ASDiv, MATH500): the fraction of repaired programs that the structure classifier from Evaluating Intermediate Reasoning labels as conceptual rather than brute-force or ungrounded.

Four checks decide the outcome. If temperature-swept on-policy drafts do not beat greedy on-policy drafts on natural-failure repair at equal pair count, the diversity hypothesis is falsified. If the spread of failure types in the training data does not predict transfer across arms, coverage is not the mechanism. If the parrot rate stays at base-model levels despite higher pass rates, training taught the model to resample rather than to correct. And if compute-matched sample-and-filter equals repair across budgets and difficulty levels, the method has no practical case, whatever it reveals about why error-injection training fails.