Escalate, Retry, or Stop: Execution-Aware Routing Inside Code Repair Loops

When a language model writes code, runs it, reads the error, and tries again, every turn costs the same price: a full call to whatever model the system was built around. Yet the errors those turns respond to differ enormously. A missing import is a different problem from a timeout on a large hidden test, and both differ from a formal proof that has failed thirty times in a row. We propose a router that sits inside the repair loop. It is a tiny classifier that reads cheap signals from each execution, such as the error class, the fraction of tests passing, the traceback length, the size of the last edit, and how failures changed since the previous turn. From these it decides one of three actions: hand the next fix to a small model, escalate to a large model, or stop. The router must cost well under a millisecond, because a router that is slow relative to a small repair turn erases its own savings.

The opportunity matters because the repair loop already contains a free, exact verifier: the tests. That makes the obvious recipe very strong. Try the cheap model, and escalate if it fails. Our core claim is narrower and testable. Execution traces predict, before the cheap attempt is spent, which failures a small model cannot fix and which loops are thrashing without progress. Using that prediction should cut wall-clock latency and tokens beyond a fixed try-cheap-then-escalate cascade, at equal pass rate. If execution features do not predict this better than the prompt alone, the idea fails, and the experiments are built to show that cleanly.

Where This Sits

Two bodies of work meet here without touching. In code generation, RoutingGen routes each problem once, before any code exists. It uses Qwen3-8B or GPT-4o to send it either to direct prompting or to a structured reasoning path, and cuts tokens by 46.37% on average. TensorOpera Router routes prompts across a pool of models with a 110M BERT classifier, including MBPP. CP-Router routes between ordinary and reasoning models using conformal prediction over answer logits, which only works for multiple-choice formats. All three decide once, from the prompt. None can recover when its first guess is wrong, and none reads an execution result. These papers motivate the routing half of our proposal; we differ by routing at every turn on execution artifacts.

On the refinement side, OpenCodeInterpreter trains models from 7B to 70B on 68,000 multi-turn execute-and-fix dialogues. With execution feedback, its 33B model lifts HumanEval Pass@1 from 79.3% to 82.9%. ICER refines code for speed with Gemini-2.5-Pro on every turn and finds that a relaxed acceptance rule beats strict monotone filtering. Adversarial Review reaches 87% on LiveCodeBench at about 4.5x the tokens of zero-shot, and names adaptive triggering as open work. These systems supply the loop we instrument. Each runs one fixed, expensive configuration per turn regardless of the error, and that uniformity is what we remove.

Two lines of evidence constrain the design. The latency literature shows that decision overhead can cancel the gains from routing. Adaptive Feeding works because its router is a linear SVM on metadata costing under 5 ms. By contrast, Feature-Dependent Cross-Connections in Multi-Path Neural Networks and MoQAE report routing modules that erase their own speedups. We therefore rule out an LLM router and adopt the Adaptive Feeding recipe of cheap metadata plus a small classifier. The second line is VERINA. There, 64 rounds of Lean compiler feedback raise proof success only from about 4.9% to about 20%, while models thrash through large budgets. This motivates a learned stop action rather than a fixed iteration cap. Rethinking Code Refinement shows a 1.3B static judge can rank code efficiency at 72.49% accuracy, so learned predictors over code states are feasible. We predict repairability, not speed, and we use execution signals that judge deliberately ignores.

Core Claim

Let $c_S$ and $c_L$ be the measured wall-clock cost of one small-model and one large-model repair turn. Let $p_S(s)$ be the probability that the small model fixes the current failing state $s$ in one turn. Consider the simplest case, where the large model is the fallback. Trying the small model first is worthwhile only when

$$c_S + (1 - p_S(s))\,c_L < c_L \quad\Longleftrightarrow\quad p_S(s) > c_S / c_L .$$

This inequality frames the question. A fixed cascade implicitly assumes every state clears the threshold $c_S/c_L$. A router gains only on states below it, where the cheap attempt is wasted, and on states where even the large model's expected success no longer justifies $c_L$, where it should stop. Our claim is that such states exist in large numbers. Algorithmic wrong-answer patterns, timeouts, and repeated unchanged failures should populate them. We also claim that a sub-millisecond classifier over execution features can find them with useful accuracy, where prompt-only features cannot.

Research Program

We first build a trajectory corpus. On training splits of MBPP-sanitized and HumanEval-style problems, and on the Code-Feedback dialogues, we run the repair loop with two models: Qwen2.5-Coder-3B (or OpenCodeInterpreter-7B) as the small model and OpenCodeInterpreter-33B or DeepSeek-V3 as the large one. At each failing state we branch and run both models for one turn, recording success, tokens, and wall-clock time on fixed hardware. This yields counterfactual labels for $p_S$ and $p_L$ at every state. For each state we extract features: exit code, exception class, assertion versus timeout versus crash, pass fraction on visible tests, traceback length, AST diff size of the last edit, turn index, and whether the failing test set changed since the previous turn. We fit gradient-boosted trees and logistic regression, and measure inference cost directly.

The policy then applies the threshold rule with a stop action. It stops when $p_L(s)$ times a per-task value falls below $c_L$, and the value is swept to trace a cost–pass-rate curve. Three settings test transfer: functional repair (HumanEval/MBPP with the -ET edge-case tests, LiveCodeBench); efficiency refinement on Effibench, using ICER's relaxed acceptance rule; and formal proof repair on VERINA. VERINA is the regime where stopping should matter most. Routers are trained per setting and also tested across settings to see whether the error-class features generalize.

Baselines

The naive reference is the large model alone in an uncapped-to-budget repair loop, which is OpenCodeInterpreter's configuration. Its reported number is 82.9% HumanEval Pass@1 with execution feedback for the 33B model (OpenCodeInterpreter). We rerun it ourselves to obtain latency and tokens, because the paper reports no cost.

The strongest competitor is the fixed escalation cascade. The small model gets $k$ turns, then the large model takes over, with $k$ tuned on the same validation set and the same budget used to tune our thresholds. It is the strongest because it already exploits the free verifier, and no reported result was found on Lacuna for it on these benchmarks. We will report its full cost–pass-rate curve over $k$ and an iteration cap. We count ourselves as beating it only if our curve dominates it, meaning at least 15% lower median wall-clock latency at matched pass rate within a 95% bootstrap interval. Router cost is counted in our latency.

The prompt-time router is a RoutingGen-style classifier that picks small or large once from the problem text. RoutingGen reports 46.37% average token savings (RoutingGen) but no number inside repair loops. We run it both with its LLM classifier and with a distilled BERT-size version so that decision cost is visible.

On VERINA, the reference is the fixed 64-round loop, which the paper reports reaches about 20% proof success (VERINA). Beating it means matching that success with substantially fewer total rounds.

The key ablation trains the identical classifier on prompt features only, then on prompt plus turn index. Any gain beyond these isolates the contribution of execution signals.

Why not just…?

Why not just use the try-cheap-then-escalate cascade? It is a real threat, possibly the main one. The inequality says it is already optimal whenever most states clear $p_S > c_S/c_L$. That is likely if the small model is strong and $c_L/c_S$ is large. We measure the distribution of $p_S$ across states directly from the counterfactual corpus. If fewer than roughly a tenth of states fall below threshold, we will report that the cascade suffices and that per-turn routing is unnecessary. That would itself be a useful negative result for the agent-design literature.

Why not just route on error type with hand-written rules? Rules such as "syntax errors to the small model" capture part of the signal at zero cost, and we include them as a baseline. The evidence they will fall short is indirect. ICER and VERINA show the costly failures are wrong-answer and non-converging states, which share an error class with easy ones. Distinguishing them needs trajectory features like unchanged failing tests. If rules match the learned router, we will ship the rules.

Why not just route once at the prompt, as RoutingGen does? A prompt-time decision cannot see that a "simple" problem's third attempt is failing the same hidden edge case. RoutingGen itself notes it cannot recover from misroutes. The prompt-only ablation settles whether execution features add predictive power, measured as AUC for $p_S$.

Why not just cap iterations? A cap handles thrashing bluntly but stops productive loops too. VERINA's gains accumulate slowly across many rounds. We compare caps at every value against the learned stop action on the same cost–success axes.

Decisive Checks

The first test is predictive. Does a classifier on execution features predict small-model repair success with AUC clearly above the prompt-only model, say a gain of at least 0.05 on held-out problems? If it does not, the idea is dead regardless of downstream numbers. The second test is end-to-end: dominance over the tuned cascade on wall-clock latency at matched pass rate on HumanEval-ET, MBPP-ET, LiveCodeBench, and Effibench, with router cost included. The third test is VERINA. Does the stop action recover roughly the 20% proof success of 64 rounds at a fraction of the rounds? We will also check for silent quality loss on the -ET edge-case tests, since routing to cheaper models could pass visible tests while degrading hidden ones.