When a coding agent builds software across a sequence of changing requirements, it has to live with its own earlier decisions. SlopCodeBench shows how badly this currently goes. Agents that extend their own code across about five successive specification checkpoints pass only 14.8% of checkpoints. Structural erosion, meaning complexity piling into already-complex functions, rises in 77% of trajectories, and agent code accumulates erosion five times faster than human code (SlopCodeBench). Video modeling faced a structurally similar problem and made real progress on it. A model that sees frames one at a time, without the future, can approach or even beat a model that sees the whole video, provided it keeps a small, well-maintained summary of the past. Online person search beats the best offline method on the Cast Search in Movies dataset, 69.08% against 67.99% mean average precision (Online Multi-modal Person Search in Videos). Streaming action segmentation, by contrast, still trails at 72.66% against 81% on GTEA (Streaming Video Temporal Action Segmentation In Real Time). The video literature therefore offers both a vocabulary and a measurement habit. It compares every online system against an offline counterpart and asks how much of the gap is unavoidable (the future carries information) and how much is fixable (the past was summarized badly).
This note proposes to bring that measurement habit to iterative software development and to test one specific fix. The measurement compares three settings: an agent that edits its own code checkpoint by checkpoint, an agent that regenerates from scratch at each checkpoint using the specification history, and an agent given every future specification up front. Together these split the failure into two parts: loss from not knowing the future, and loss from inheriting one's own degraded code. The fix is a small team of agents in which a dedicated maintainer role keeps a fixed-size design ledger. The ledger is a compact written record of interfaces, invariants, and known debts, analogous to the memory tokens of online video models. The maintainer is triggered to refactor only when measured erosion crosses a threshold, and it rolls back any edit that breaks previously passing tests.
Core Claim
Our hypothesis is that most of the online-offline gap in multi-checkpoint coding comes from self-inflicted erosion, not from missing foresight. If that is true, a bounded, explicitly maintained design memory owned by a separate role should close most of the gap without growing the context with history. Written as a decomposition, let $P_{\text{inc}}$, $P_{\text{regen}}$, and $P_{\text{fore}}$ be checkpoint pass rates for the incremental, regenerate-from-scratch, and full-foresight agents. Then
$$ P_{\text{fore}} - P_{\text{inc}} = \underbrace{(P_{\text{fore}} - P_{\text{regen}})}_{\text{foresight loss}} + \underbrace{(P_{\text{regen}} - P_{\text{inc}})}_{\text{inheritance loss}}, $$
where the second term measures how much worse an agent does by building on its own prior code than by starting clean with the same information. The decomposition is an identity. The empirical content lies in the sizes of the two terms and in whether our method moves $P_{\text{inc}}$ toward $P_{\text{fore}}$ while keeping the per-checkpoint context bounded. As far as we can find in the research file, no prior work measures this split. SlopCodeBench documents erosion but proposes no architecture and runs no foresight or regeneration controls. Multi-agent coding systems such as MAGIS and Self-collaboration Code Generation via ChatGPT operate on single static tasks, and the latter's shared blackboard grows linearly with turns.
Research Program
The work has two parts: a measurement study and a method. The measurement study runs all three reference agents on SlopCodeBench under a matched model and token budget. It reports checkpoint pass rate together with the benchmark's own erosion and verbosity scores, so that the decomposition above is estimated per problem and per checkpoint depth. We expect inheritance loss to grow with checkpoint index, since erosion compounds. Foresight loss should concentrate on checkpoints whose specifications break an earlier interface. This parallels the observation that streaming action segmentation lags most where future context carries structural cues.
The method borrows three roles from the literature, each for a stated reason. The first is a Developer agent that writes localized edits, as the Developer agents in MAGIS write small patches rather than whole files. The second is a Maintainer agent that owns the design ledger. The ledger is a capped-length text document, on the order of a few thousand tokens, listing public interfaces, data formats, invariants, and a debt list of functions flagged as eroding. Each checkpoint, the Developer sees only the new specification, the ledger, and the files the Maintainer points it to. It never sees the full edit history. This mirrors two things. MAGIS's Repository Custodian keeps the repository out of other agents' context. Transeger conditions each new segment on short text descriptions of preceding segments rather than raw past frames. Like the memory in InstanceFormer or ARFlow, the ledger is rewritten each step, not appended, so the per-checkpoint context stays roughly constant regardless of how many checkpoints have passed (InstanceFormer, ARFlow).
The third ingredient is a trigger-and-rollback discipline. After each Developer edit, cheap static measurements (cyclomatic complexity concentration and duplication, the same quantities SlopCodeBench scores) are computed. The Maintainer runs a refactoring pass only when these cross a threshold. The reason to trigger rather than always refactor comes from MAgICoRe, which found that applying refinement to every math problem cost 5.2% while routing only hard cases to refinement helped. Every refactor is tested against the accumulated test suite and reverted if any previously passing test fails. This is the rollback mechanism of Fang et al. for hardware code, used here specifically to prevent the over-editing regressions that Adversarial Review reports when reviewers act aggressively. Refinement budgets per checkpoint are capped at a few rounds, since Yan et al. find execution-feedback gains plateau after two to four turns.
Two constraints from prior work shape the design. Self-collaboration Code Generation via ChatGPT reports that models under roughly 13B parameters fail to hold distinct roles. We therefore run the main study on a frontier model and treat a small open model only as a secondary robustness check. Srivastava et al. warn that loops optimizing only for passing tests can amplify security flaws, so refactors are also scanned with a static security checker and flagged findings are reported alongside pass rate.
Baselines
The naive reference point is the plain incremental agent. A single agent receives each new checkpoint specification along with its current codebase and edits until tests pass or a round cap is hit. This is the setting SlopCodeBench evaluates. The strongest reported number is the best agent's 14.8% checkpoint pass rate (SlopCodeBench). We rerun it under our model and budget rather than relying on the published figure alone, so the comparison uses identical conditions. Beating it means a statistically reliable gain in checkpoint pass rate together with lower erosion and verbosity slopes.
The strongest existing method is Adversarial Review applied at every checkpoint. A Main Agent edits the code, while a Reviewer and a Critic, who must categorize disagreements and cite line numbers, debate over the frozen artifact before the Main Agent revises (Adversarial Review). It is the strongest competitor because it is a lean three-agent refinement loop with published gains on repository-level repair: 75.2% on SWE-bench Verified against 71.6% zero-shot and 72.6% for the five-agent MARS system, and 87% on LiveCodeBench. We found no reported result on Lacuna for Adversarial Review on SlopCodeBench. We run it with the same backbone model and cap its tokens to match our method's total spend. Adversarial Review reports about 4.5 times zero-shot tokens, so matching in both directions matters. Our method beats it if it reaches equal or higher checkpoint pass rate with lower erosion at equal tokens, or equal pass rate at clearly fewer tokens.
The regenerate-from-scratch agent serves both as a measurement control and as a genuine competitor. At each checkpoint it writes the program anew from the full specification history, discarding prior code. The full-foresight agent receives all checkpoint specifications at once and is then scored checkpoint by checkpoint. It is not a deployable baseline but the offline ceiling in the parity comparison.
Three ablations isolate the key ingredients. The first gives the same single agent the same ledger and the same refactor trigger, but with no separate Maintainer role. This tests whether role separation matters or whether the ledger and trigger alone do the work. The second keeps the roles but gives the Developer full edit history instead of the capped ledger, which tests whether bounded memory helps or merely saves tokens. The third removes rollback.
Why not just…?
Why not just regenerate from scratch at every checkpoint? This is the most serious threat, and the experiment is built to settle it. If $P_{\text{regen}}$ matches our method's pass rate at comparable cost, the right conclusion is that coding agents should not maintain code at all under black-box specifications, and we will report that. Two things could make regeneration fall short. Its context grows with the specification history, so cost per checkpoint rises with depth. It may also silently break unstated continuity such as persisted file formats or output ordering that later checkpoints depend on. We measure both directly, tracking tokens per checkpoint against depth and counting failures attributable to changed interfaces. We do not assume either will occur.
Why not just tell the agent to write clean code? SlopCodeBench already tested quality-aware prompts. They reduce initial verbosity and erosion but increase cost per checkpoint and slightly decrease correctness (SlopCodeBench). A one-time instruction at the start does not address the compounding across checkpoints. We still include this prompt variant as a cheap baseline so its erosion slope can be compared directly with ours.
Why not just add a periodic refactoring pass by the same agent? This is the first ablation above, and it could well work as well as the full method. If it does, the multi-agent part of the proposal is unnecessary, and the contribution reduces to the ledger and the erosion-triggered, rollback-guarded refactor. Confirmation bias in self-review is the usual argument for separate roles (Self-collaboration Code Generation via ChatGPT). We treat that argument as a hypothesis to test, not a premise.
Why not just run a stronger reviewer loop? Reviewers inspect the current diff, not the trajectory. Adversarial Review itself reports scope creep that breaks previously passing tests. The per-checkpoint Adversarial Review baseline measures whether review alone slows the erosion slope. We expect it to improve early checkpoints and lose ground later, but the data will decide.
Proposed Experiments
The primary benchmark is SlopCodeBench: 36 problems and 196 checkpoints, scored on checkpoint pass rate, the benchmark's structural erosion and verbosity measures, and the fraction of problems solved end to end, which no published agent achieves. Every condition uses the same backbone model and a matched per-problem token budget, with three seeds and problem-level bootstrap confidence intervals.
The first decisive check is the decomposition itself. We estimate foresight and inheritance loss as functions of checkpoint depth. If inheritance loss is small, the premise fails and the method is not worth pursuing. The second check asks whether our method recovers a meaningful fraction of inheritance loss, reported as $(P_{\text{ours}} - P_{\text{inc}})/(P_{\text{regen}} - P_{\text{inc}})$, while keeping per-checkpoint context flat with depth. The third check uses the ablations to test whether the separate Maintainer role and the capped ledger each contribute beyond a single agent with the same tools. The fourth is a stress test that reorders SlopCodeBench checkpoints where dependencies allow, or truncates problems early. Reordering and truncation vary how much future information matters, so foresight loss should respond to them while inheritance loss should not, which validates that the decomposition measures what it claims.
As a secondary sanity check, we run the method in single-checkpoint mode on SWE-bench Verified. There the ledger has nothing to remember, and performance should roughly match plain Adversarial Review (75.2% reported), showing the machinery does not hurt one-shot repair. A repeat of the main study with a small open model tests the role-adherence constraint reported for sub-13B models.
References
- SlopCodeBench: Measuring Code Erosion Under Iterative Specification Refinement
- Online Multi-modal Person Search in Videos
- Streaming Video Temporal Action Segmentation in Real Time
- MAGIS: LLM-Based Multi-Agent Framework for GitHub Issue Resolution
- Self-Collaboration Code Generation via ChatGPT
- InstanceFormer: An Online Video Instance Segmentation Framework
- ARFlow: Auto-regressive Optical Flow Estimation for Arbitrary-Length Videos via Progressive Next-Frame Forecasting
- MAgICoRe: Multi-Agent, Iterative, Coarse-to-Fine Refinement for Reasoning
- MEIC: Re-thinking RTL Debug Automation using LLMs
- Adversarial Review: Structured Disagreement for Grounded Agentic Code Review
- CodeTransOcean: A Comprehensive Multilingual Benchmark for Code Translation
- Do Users Write More Insecure Code with AI Assistants?