A language model fixing code has two ways to spend more effort. It can go wider: sample many candidate programs in parallel and let them vote on behavior. Or it can go deeper: run the current program, show the model the failing test and error message, and ask for a revision. Width is cheap in wall-clock time because the samples run side by side. Depth is expensive because each round must wait for the previous one. Current systems fix one mix of the two for every problem. Some always sample 64 programs and vote. Others always run three debugging rounds. Others always run a full committee of patch reviewers.
This note proposes a scheduler that makes the choice per problem and per round. The signal it uses is the structure of disagreement among the samples it already has. The intuition comes from image reconstruction, where averaging several noisy generative samples cancels errors that are random and independent but cannot remove errors that every sample shares. Applied to code, this gives a simple rule. When samples disagree, the failure is "noise-like" and more width plus a vote should help. When samples agree with each other but fail an available test, the failure is "bias-like": the model is consistently wrong, more samples will only repeat the mistake, and execution feedback (depth) is the only lever.
Core Claim
The claim is that the ratio of noise-like to bias-like error in a pool of code candidates can be read cheaply from two observable quantities. One is how concentrated the candidates are in behavior. The other is whether the dominant behavior passes the tests the system is allowed to see. We further claim that routing each round to width or depth according to this reading reaches the accuracy of fixed large-budget voting at substantially lower wall-clock latency. The claim is falsifiable at the instance level. If widening does not help more on high-disagreement instances than on consistent-failure instances, and deepening does not show the reverse pattern, the idea is wrong regardless of aggregate numbers.
The error-cancellation intuition is standard in continuous domains. Perception-Distortion Trade-off in the SR Space Spanned by Flow Models shows that averaging flow-model samples moves the output toward the conditional mean and raises fidelity. Diffusion reconstruction work in MRI (DiffCMR), PET (Contrastive Diffusion Model with Auxiliary Guidance) and ECG denoising (DeScoD-ECG) averages multiple sampling runs to suppress sampling variance. All of these use a uniform number of runs, and all average in a continuous space. We borrow only their error model, not their averaging operator.
For code, the discrete analogue of averaging is functional voting. Programs are grouped by their outputs on shared inputs. The operator comes from Majority Voting For Code Generation, which executes N sampled programs, clusters them by execution vector, and selects the largest agreeing cluster. That paper also reports the failure our scheduler targets. Its test-time training raises mean accuracy but not best@64, because incorrect programs can agree on superficial cases (a "Lucky Hit"). In error-model terms this is shared bias, which voting cannot cancel.
The Scheduler
At round $t$ the system holds a pool of candidates and runs them on two kinds of inputs. The first kind is the problem's visible example tests. The second is a set of model-generated inputs, which have no reference outputs and are used only to measure agreement. Let $a_t$ be the fraction of candidates in the largest behavioral cluster and $v_t$ the fraction of visible tests that cluster passes. These two numbers place the round in one of four regimes.
When $a_t$ is high and $v_t = 1$, the scheduler stops. When $a_t$ is low and some cluster passes the visible tests, the scheduler widens, drawing more samples and re-voting. This is the variance regime. When $a_t$ is high and $v_t < 1$, the scheduler deepens. It sends the consensus program, its failing test and its traceback back to the model for revision. This is the consistent-error regime. When $a_t$ is low and every cluster fails, the scheduler deepens from one representative per cluster in parallel. The pool's diversity then serves as a set of distinct repair starting points rather than something to average away.
Visible-test failures are ground truth, not model self-assessment. This is what separates the scheduler from probability-based stopping rules, which break on confident wrong answers.
The objective is wall-clock latency rather than token count, because the two trade off differently. With $P$ parallel decoding slots, round $t$ drawing $K_t$ samples costs roughly
$$ L = \sum_{t=1}^{T} \left( \left\lceil K_t / P \right\rceil \, c_{\text{dec}} + c_{\text{exec}} \right), $$
where $c_{\text{dec}}$ is the time to decode one batch of candidates and $c_{\text{exec}}$ is the time to run tests in the sandbox. Under this cost, width is nearly free up to $P$, and every extra round of depth pays the full sequential cost. The regime thresholds on $a_t$ and $v_t$ are the only fitted parameters. They are fitted by grid search on a few hundred calibration problems under a target latency budget.
As a further step, we will replace the thresholds with a small classifier over the same two features plus round index and pool size. This checks whether anything is lost by the hand-set regimes. The scheduler is black-box and needs no model weights.
Research Program
The work has three parts. The first is a diagnostic study. On a fixed open coding model, for every problem we record the round-one pool, $a_1$ and $v_1$, and then run both a pure-width continuation and a pure-depth continuation to a common latency. This measures directly whether the regime predicts which lever helps. It is the decisive empirical fact, and it costs nothing beyond logging.
The second part builds and tunes the scheduler on competitive programming, where behavioral clustering is reliable.
The third part transfers to repository-level repair on SWE-bench. There, candidates are patches, behavior is measured on reproduction tests the model writes, and "visible tests" are the issue's reproduction script plus the existing tests that passed before the patch. This last setting is harder because behavioral clusters are noisier. We treat it as a stress test rather than the main result.
The proposal sits among three lines of prior work. The first allocates compute by difficulty. Learning How Hard to Think trains probes on hidden states to predict the gain from extra samples, halving samples on TACO. Escape Sky-high Cost stops self-consistency when a sliding window of answers agrees. Thought calibration stops reasoning with calibrated linear probes. Adaptive Generate-Rank-Verify doubles generation and verification budgets geometrically under a monotone reward score. All four decide how much to sample. None decides whether to sample or to revise, and none reads execution tracebacks. Early stopping by agreement is, in our terms, correct only in the variance regime: it cannot distinguish unanimous-right from unanimous-wrong.
The second line runs fixed repair loops. CodeTransOcean runs three rounds of compiler-feedback debugging and reports gains that flatten after the first round. MAGIS runs a fixed four-agent team. REFINE samples a fixed pool of diverse patches and merges them with reviewer and aggregator agents at about $6.59 per issue. These supply the depth and patch-aggregation operators but spend them uniformly. The CodeTransOcean plateau motivates our claim that depth should be spent only where error is bias-like.
The third line removes iteration to cut latency. SWE-Fixer uses exactly two model calls. Meta-Reflection internalizes reflection into a single forward pass. These constrain the design: any scheduler must beat them on latency-adjusted accuracy, not accuracy alone.
Baselines
The naive reference points are a single greedy sample, and a single sample followed by a fixed three rounds of traceback-guided self-debugging, in the style of CodeTransOcean. These show what pure depth buys at fixed cost.
The strongest existing competitor on competitive programming is functional majority voting at a fixed budget of N=64, as in Majority Voting For Code Generation. It is the strongest because it already implements the discrete aggregation operator and is fully parallel, so its latency is low when slots are plentiful. That paper reports LiveCodeBench-v6 accuracy for Qwen3-4B-Thinking-2507 rising from 37.7% to 52.7% with GenCluster plus functional voting. We will reproduce this with the same model, the same problems, the same input generator and the same sandbox, and report our reproduced number beside the published one. Beating it means reaching at least our reproduced accuracy at no more than half its median wall-clock latency under the same $P$, or exceeding it at equal latency.
The adaptive-allocation competitors are three. The first is Adaptive Generate-Rank-Verify. It reports being 5.5x cheaper than the best fixed-budget policy at reaching full success on solvable LiveCodeBench problems, but the file reports no accuracy figure for it at a fixed budget. The second is the hidden-state probe of Learning How Hard to Think, which reports a 50% sample reduction at matched best-of-k on TACO. We will train its probe on the same calibration problems we use for threshold fitting. The file notes that this probe normally needs 3,000 or more labeled instances, so we will also report it at that larger size to avoid starving it. The third is agreement-based early stopping of Escape Sky-high Cost, applied to execution-vector clusters instead of text answers.
All adaptive methods get the same model, the same maximum sample budget, the same calibration set and the same grid-search budget for their hyperparameters.
Three ablations isolate the key ingredient. Width-only adaptivity is early stopping on functional agreement with depth disabled. Depth-only adaptivity revises the top cluster whenever it fails a visible test, with width fixed. Random routing keeps our stop rule but chooses width or depth by coin flip. An oracle router, which sees which lever eventually succeeds, gives the ceiling.
On SWE-bench Lite, the reference is REFINE, which reports 51.67% with AutoCodeRover at about $6.59 per issue. The file lists AutoCodeRover's own baseline both as 37.0% (REFINE) and as 16.11% (MAGIS), so we will rerun the base agent ourselves rather than trust either figure. The two-call SWE-Fixer, reported at 24.7% on Lite and 32.8% on Verified, anchors the low-latency end.
Why not just…?
Why not just stop sampling once the functional clusters agree? This is the most serious threat. Agreement-based early stopping plausibly captures most savings on easy problems, and easy problems dominate averages. The Escape Sky-high Cost authors themselves flag consistent hallucination as its failure mode, and the "Lucky Hit" result in Majority Voting For Code Generation shows agreement on wrong programs is common in code. The width-only ablation settles this directly. If our scheduler's gain over it comes only from instances where round one already had high agreement and passed the visible tests, the depth branch adds nothing and we will report that. The proposal survives only if the consistent-error regime is frequent enough, and deepening there fixes enough of it, to move the latency–accuracy curve.
Why not just predict difficulty from hidden states before generating anything? A probe such as the one in Learning How Hard to Think decides a budget before any program is run. It therefore cannot know why a problem is hard, and it outputs only a sample count, not a choice between sampling and revising. It also needs white-box access and thousands of labeled problems. We run it as a baseline, and we also combine it with our scheduler as a prior on the round-one pool size. If the combination beats our scheduler alone, the two signals are complementary. If the probe alone matches us, the execution-based regime reading is unnecessary.
Why not just run a fixed small number of debugging rounds, or collapse to two calls? Fixed debugging spends sequential latency on every problem. CodeTransOcean shows the gains are small and front-loaded: 48.57% rising to 52.57% after three rounds on its LLMTrans subset. The two-call design of SWE-Fixer avoids that cost but cannot recover when its patch fails. Both baselines appear on our latency–accuracy plot. The scheduler must dominate them there, not merely beat them in raw accuracy.
Why not just use a reward model's score to decide, as in generate-rank-verify? Adaptive Generate-Rank-Verify relies on scores being monotone in correctness. It never inspects failures and never revises. A consistently wrong cluster can score highly, and a candidate with a trivially fixable error can score low. We run it at matched budget. If it matches our scheduler, the extra structure is not needed.
Proposed Experiments
The main testbed is LiveCodeBench-v6 with Qwen3-4B-Thinking-2507, matching the functional-voting paper. We add a second open coder on TACO to compare against the hidden-state probe, and MBPP as a small sanity set. Meta-Reflection reports 63.4% pass@1 on MBPP for its method, which serves only as a single-pass latency reference. Metrics are accuracy on hidden tests, median and 90th-percentile wall-clock latency at $P \in \{8, 32\}$, generated tokens, and sandbox calls. Every method is plotted as a latency–accuracy curve by sweeping its budget knob.
The decisive check is the interaction test from the diagnostic study. We bin problems by the round-one regime and measure the accuracy gain per second of latency from pure width versus pure depth in each bin. The hypothesis predicts a crossover: width wins in the low-agreement bin, depth wins in the high-agreement-but-failing bin. A flat pattern falsifies the error model and with it the scheduler.
The second check asks whether visible tests are too weak to detect consistent error. We measure how often a cluster passes every visible test but fails hidden tests. If this is common, the stop rule is unsafe, and we add generated tests validated by agreement, reporting the latency cost.
The third check transfers the fitted thresholds, unchanged, to SWE-bench Lite and Verified with patches as candidates. It asks whether regime frequencies and the crossover persist when behavioral clustering relies on model-written reproduction tests. We compare against a rerun of the base agent, the REFINE-style fixed patch pool, and the two-call pipeline, reporting resolution rate against wall-clock time and dollar cost per issue.
References
- Perception-Distortion Trade-Off in the SR Space Spanned by Flow Models
- DiffCMR: Fast Cardiac MRI Reconstruction with Diffusion Probabilistic Models
- Contrastive Diffusion Model with Auxiliary Guidance for Coarse-to-Fine PET Reconstruction
- DeScoD-ECG: Deep Score-Based Diffusion Model for ECG Baseline Wander and Noise Removal
- Majority Voting For Code Generation
- Learning How Hard to Think: Input-Adaptive Allocation of LM Computation
- Escape Sky-high Cost: Early-stopping Self-Consistency for Multi-step Reasoning
- Thought calibration: Efficient and confident test-time scaling
- Adaptive Generate-Rank-Verify: Inference-Time Search with Costly Verification
- CodeTransOcean: A Comprehensive Multilingual Benchmark for Code Translation
- MAGIS: LLM-Based Multi-Agent Framework for GitHub Issue Resolution
- REFINE: Enhancing Program Repair Agents through Context-Aware Patch Refinement
- SWE-Fixer: Training Open-Source LLMs for Effective and Efficient GitHub Issue Resolution
- Meta-Reflection: A Feedback-Free Reflection Learning Framework