Refining Code Against a Moving Evaluator

When a language model is asked to improve its own code over several rounds, it is graded by the same small set of tests on the same machine every time. Like any optimizer pointed at a fixed target, it learns to please that target. It produces code that is fast on the seen inputs but slow or wrong elsewhere. It also produces code that passes this round's tests while piling up duplicated, tangled logic that breaks the next extension. Both failures are documented. Strict "only accept faster code" loops overfit to the seen unit tests (Improving Code Efficiency with Iterative Refinement using LLMs). Agents that extend their own code grow structurally eroded in 77% of trajectories, even when prompted to care about quality (SlopCodeBench).

We propose to borrow the main lesson of embodied learning, that breadth of environments beats depth of repetition (Data Scaling Laws in Imitation Learning for Robotic Manipulation, Robot Utility Models). We apply it to the evaluator inside the refinement loop. Each candidate edit is scored separately on correctness, runtime, memory and maintainability. Those scores are measured on a freshly sampled set of execution environments at every round: new input distributions and sizes, resource caps, and interpreter and library versions. Edits are accepted only if no score regresses on that fresh sample. In two sentences: refinement loops overfit because their evaluator never changes, so we resample the evaluator each round and report per-dimension feedback instead of one number. We predict this yields code that keeps its gains on held-out environments and on future specification changes, at equal execution budget.

Core Claim

The claim is that environmental breadth matters during refinement and not only during training. Take a fixed budget of execution calls per round. Spending it on many freshly drawn environments should give better held-out generalization than spending it on repeated measurements in a fixed environment set of the same size. The improvement should show on both efficiency and maintainability. Decomposing feedback by dimension should matter too: when scores are collapsed into a weighted scalar, the loop can trade a regression on one axis for a gain on another. This is the "optimization cheating" failure that motivates decomposition in video quality work such as DOVER, here transferred to code.

The closest prior systems each lack one piece of this. ICER refines efficiency with execution feedback and fixes overfitting by relaxing the acceptance rule. We keep its relaxed policy but attack the cause, a fixed evaluator. SlopCodeBench measures erosion but gives agents only pass/fail feedback. RACE decomposes quality into four axes but only in a one-shot static setting. VeriEnv and TDScaling scale environment or trajectory diversity for training agents, but neither varies the environment used to judge an individual program during its own refinement.

Research Program

We will build an environment sampler for Python code tasks. An environment is a tuple of four things: an input generator, a resource limit, a runtime (CPython minor versions, with or without common library versions), and a timing noise model. The input generator is drawn from a family that covers scale, skew, adversarial worst cases and edge structures. The resource limit sets memory and time caps. The timing noise model uses repeated runs to estimate measurement variance.

For EffiBench problems, input generators come from LLM-written generators validated against the canonical solution. For SlopCodeBench, each checkpoint's contract tests are run under the sampled runtimes and resource caps. One part of the environment space is held out entirely and never touched during refinement: specific runtimes, input families and memory caps.

Feedback is a vector with one entry per dimension. Correctness, runtime and peak memory come only from execution. Maintainability uses static metrics (cyclomatic complexity, duplication, the erosion measure from SlopCodeBench), complemented by an LLM judge. The judge does only one narrow job: given the previous and proposed diff, it says whether the change concentrates complexity or duplicates logic, and where.

This boundary follows the evidence that LLM judges are poor at checking correctness. Function-Level Execution Feedback reports about 52% agreement with execution on passing functions. Pareto Optimal Code Generation reports blindness to small near-miss bugs. The judge is run with both diff orderings, to address the position bias reported in Justice or Prejudice?. We validate it against a ground truth that SlopCodeBench supplies for free: whether the next checkpoint's extension succeeds.

The acceptance rule works as follows. Let $c$ be the current program and $c'$ a candidate. Let $E_t$ be a fresh sample of $k$ environments at round $t$. Let $\Delta_d(e)$ be the change in dimension $d$ in environment $e$, signed so that positive means better, and let $\tau_d$ be the measured noise floor for that dimension. Then $c'$ is accepted when

$$\min_d \; \mathrm{median}_{e \in E_t}\, \Delta_d(e) \ge -\tau_d \quad\text{and}\quad \max_d \; \mathrm{median}_{e \in E_t}\, \Delta_d(e) > \tau_d .$$

That is, no dimension gets worse beyond noise, and at least one gets better. Correctness is a hard constraint across all of $E_t$. The model receives the per-dimension, per-environment breakdown as text, focused on the dimension and environment that regressed most. Its critique is therefore localized, which addresses the concern that long lists of upfront constraints hit RACE's "complexity wall."

Baselines

The naive reference is one-shot generation followed by plain pass/fail iterative repair on the provided tests. This is the setting in which SlopCodeBench reports a best checkpoint pass rate of 14.8%, and in which GPT-4 code on EffiBench has an average normalized execution time of 3.12 (EffiBench). It shows how much any feedback beyond correctness buys.

The strongest existing competitor for efficiency is ICER's relaxed update policy, which accepts any functionally correct candidate. It is the most direct prior answer to efficiency overfitting. The reported figure is that Gemini-2.5-Pro reduced the normalized runtime of DeepSeek-R1-Distill-Qwen-32B code on EffiBench from 1.89x to 1.03x of canonical. We will rerun ICER with the same generator model, the same number of rounds, and an identical total execution-call and token budget. We will report its normalized time both on its own seen tests and on our held-out environments.

Beating it means two things. On held-out environments, we must achieve a statistically lower normalized runtime and memory, with a correctness rate no worse than ICER's. On seen tests, we must be within noise of ICER's runtime. If we only match ICER on seen tests and lose nothing elsewhere, that is not a win.

For maintainability, the strongest competitor is SlopCodeBench's own quality-aware prompting, run at equal cost per checkpoint. A second strong competitor is staged verification from Pareto Optimal Code Generation, used as a selection method over the same number of samples.

Three ablations isolate the key ingredients. The first keeps the vector feedback but uses a fixed environment set of size $k$ drawn once, so the comparison is breadth versus repetition at equal calls. The second keeps resampling but collapses the vector into a tuned weighted scalar, which tests whether decomposition matters. The third removes the LLM maintainability judge and leaves only static metrics, which tests whether the semantic judge adds anything. Each arm receives the same tuning budget for its free parameters.

Why not just…?

Why not just use many more test inputs in a single environment? This is the most serious threat. Much of what we call environmental diversity may simply be input diversity. If held-out generalization comes entirely from varied input generators, then the runtimes and resource caps are decoration and the claim shrinks to "resample inputs each round." We will settle this with an arm that varies only inputs and one that varies only runtime and resource limits, both at equal call count. If the input-only arm matches the full sampler on every held-out axis, we will report the narrower claim. We will then drop the runtime-variation machinery from the method rather than defend it.

Why not just collapse everything into one weighted score and refine against that? Scalar objectives are cheaper and easier to tune. But they allow exactly the trade the decomposition literature warns about: a memory blowup paid for with a runtime gain, or duplicated code that buys a passing test. EffiBench's worst cases illustrate how extreme such hidden costs get, with GPT-4 code up to 13.89x slower and 43.92x more memory-hungry. The scalar ablation will show whether such trades actually occur in our loops, measured as the fraction of accepted edits that regress some dimension on held-out environments.

Why not just ask a strong LLM to review and score the code in one pass? For correctness the evidence is against it: about 52% agreement with execution in Function-Level Execution Feedback, and the small-bug blind spot reported in Pareto Optimal Code Generation. Maintainability is less clear, so we test the judge directly. We will check whether its "this edit erodes structure" verdict predicts failure on the next SlopCodeBench checkpoint better than static metrics alone. If it does not, it is removed.

Why not just prompt for quality upfront? SlopCodeBench reports that quality-aware prompts reduce initial erosion, but they raise cost and slightly lower correctness, and erosion still appears later. RACE reports that compliance collapses once five or more simultaneous constraints are imposed. We include this arm at equal cost and expect it to lose on later checkpoints. That is an empirical bet, not an assumption.

Proposed Experiments

On EffiBench, we will refine solutions from two open generator models for a fixed number of rounds under every arm. We report correctness, normalized execution time and normalized memory on held-out environments. We also report the gap between seen-test and held-out performance, which directly measures overfitting to the evaluator.

The decisive check is a scaling curve. We vary the number of distinct environments sampled per round while holding total execution calls fixed. Embodied results such as Data Scaling Laws in Imitation Learning for Robotic Manipulation suggest held-out gains should rise with breadth and flatten, whereas extra repetition in a fixed set should yield little held-out gain. A flat curve would falsify the core claim.

On SlopCodeBench, we will run the full multi-checkpoint protocol. We report checkpoint pass rate against the 14.8% reference, together with erosion and verbosity growth per checkpoint. The key test is whether resampled, decomposed feedback slows erosion without lowering pass rate. Quality prompting failed this test in the original paper.

On RACE, we report the four-axis scores after refinement. This checks that gains do not come from sacrificing readability.

Across all three benchmarks, we will log the LLM judge's agreement with next-checkpoint outcomes and with swapped-order versions of itself. Total cost will be reported in execution calls and tokens, so the budget comparisons can be audited.