AI agents that write formally verified code often get stuck because they keep proving the wrong program, even when they could have proved a right one. An agent writes an implementation, attempts a machine-checked proof that it meets its specification, and fails. It then spends the rest of its budget patching the proof, even when the real problem is that the code was built in a shape no clean proof can follow. Journal editorial boards handle a closely analogous problem. An author defends their manuscript, a referee finds a flaw, and someone has to decide whether the flaw calls for a revision or a rejection. Boards give that decision to an associate editor. This editor reads the submission and the referee reports but did not do the work and does not see the author's private reasoning. We propose to test whether verified code generation improves when the decision to throw away an implementation and replan is made by a separate arbiter agent whose view is deliberately restricted. The arbiter would see only the specification, a short plan sketch, and the verifier's summary of the proof goals that would not close. It would not see the code or the author's reasoning trace.
The claim is narrow and falsifiable. Most of the value of "hierarchy" in multi-agent coding should come from information restriction at the decision point, not from adding agents. If that is true, a restricted-view arbiter should beat a full-context arbiter, a fixed restart rule, and a single agent that plans proof and program jointly, all at equal compute. If it is false, the editorial analogy is decoration, and the experiments will show that too.
Why this question, and what already exists
The specification-implementation gap is the distance between what a system is supposed to do and what its code does. In verified code generation it now shows up mainly as a proof bottleneck. VERINA benchmarks 189 Lean 4 tasks (Lean is a proof assistant in which programs and correctness proofs are checked by the same compiler). It reports that OpenAI o3 reaches 72.6% pass@1 on code generation but only 4.9% on proof generation and 3.2% end to end. Even 64 rounds of compiler feedback raise proof success only to about 20%. Vero scales this to 43 repository-level instances and makes an observation that motivates this proposal: agents rarely refactor code when a proof gets stuck, and treat the implementation as fixed scaffolding. Vero also finds that full solves needed about twice the proof text of unfinished runs while implementation size stayed similar. Effort goes into proofs, not into reshaping code.
P³ is the closest existing work and supplies part of our method. It has one agent plan a program sketch together with a proof sketch. It separates local elaboration errors from plan-level errors and mandates a retreat to replanning for the latter. It reports gains of 4.6 to 11.2 percentage points over the strongest baselines, and 3.3 to 8.3 points from joint planning over implementation-only planning. P³ leaves the judgment "is this plan-level?" to the same agent that wrote the code. That is exactly where we expect bias. VERINA reports that giving a model the implementation degrades its specification generation, which suggests that implementation detail distorts judgments that should depend only on the specification. This evidence constrains our design: the arbiter must not see the code.
The editorial-board literature supplies the structural idea rather than a method. Journal front matter such as IEEE Transactions on Cybernetics publication information describes boards as cascading tiers. Executive editors fix scope, associate editors arbitrate between referees and authors, and referees do local checks. Principal-agent analyses such as Dong's study of delegation chains explain why: a delegated agent optimizes its local objective, here "close this proof goal," instead of the institutional one, here "produce a verified repository." Two multi-agent results constrain which structures are worth trying. CooperBench shows that flat peer pairs of coding agents reach about 25% success, roughly 50% below a single agent doing both tasks, with divergent architecture behind 29.7% of failures. Embodied LLM Agents Learn to Cooperate in Organized Teams shows that designated leadership helps, for example a 9.76% completion-time improvement for three GPT-3.5 agents, but only in household simulation and with no verification. SGCR uses an aggregator over specification-partitioned reviewers for code review, but against informal rules and without proofs. None of these tests whether what the decision-maker sees matters.
Core Claim
In verified code generation, deciding whether to repair a proof or replan the program is the step where single agents fail most. That decision is made better by an agent that sees the specification, the plan sketch, and summaries of the failing proof goals than by an agent that also sees the code and its history. Giving that agent binding authority, so that a "replan" verdict discards the code, adds value beyond advisory feedback. Both parts are testable separately.
Research Program
The system has three roles running on the same base model. An author agent produces a program-and-proof plan in the style of P³ and elaborates it in Lean 4. A verifier role is mostly mechanical. It runs the Lean compiler and condenses unclosed goals into short summaries that state which lemma is stuck, which induction it attempted, and which hypothesis is missing, with no code excerpts beyond the goal statements. Whenever the author exhausts a fixed local repair allowance, an arbiter receives the frozen specification, the current plan sketch, and the goal summaries. It returns one of two binding verdicts: continue local repair, or reject the plan and write a short reason that the author must address in a new plan. At repository scale, as in Vero, there is one arbiter per module. A top-level step freezes inter-module lemma statements so that authors cannot change them; this targets the semantic divergence CooperBench documents. The repository-level component is secondary and will be reported separately so it cannot inflate the main result.
The first contribution is a diagnostic dataset of stuck checkpoints. We run the single-agent baseline on Vero, Lean4Commit0, and VERINA and save the state each time local repair stalls. From each checkpoint we branch two continuations at matched budget, one that keeps repairing and one that replans from the specification. We label the checkpoint "plan-level" if only the replan branch solves, "local" if only the repair branch solves, and ambiguous otherwise, using several seeds per branch. This gives ground truth for the decision itself. Any decider can then be scored offline on accuracy, and on the expected solves it recovers, before any end-to-end run. This dataset is useful independently of whether our arbiter wins.
Baselines
The naive reference point is a single agent in Vero's code-and-proof mode with plain compiler-feedback repair. For context, Vero reports that GPT-5.5 (xhigh) fully solves 27 of 43 instances at $106 per solve, and VERINA reports 3.2% end to end for o3. We will rerun this baseline with our own base model rather than borrow those figures.
The strongest existing method is P³, run from its released prompting scheme on the same base model and the same token and wall-clock budget, with its own plan-level retreat rule intact. The file gives P³'s relative gain of 4.6 to 11.2 points over prior baselines but no absolute solve rate. No reported absolute result was found on Lacuna for P³ on Vero or Lean4Commit0, so we will establish its number ourselves. Beating it means a statistically reliable improvement in full-solve rate on Vero and solve rate on Lean4Commit0 at equal total tokens, counting the arbiter's tokens against our method, with no worse cost per solve.
A fixed restart rule discards code and replans after $k$ failed repair rounds, with $k$ tuned on a held-out split using the same tuning budget we give the arbiter's repair allowance. It tests whether any judgment is needed at all.
The ablation that isolates the key ingredient is a full-context arbiter. It is identical to ours in prompt, authority, and budget but also sees the code and the author's trace. A second ablation is an advisory arbiter that sees the restricted view but whose verdict the author may ignore. It separates the effect of information restriction from the effect of authority. A flat two-author peer setup in the CooperBench style is included only at repository scale, to confirm that splitting work without an arbiter hurts there as well.
Why not just…?
Why not just give a single agent more compute or more samples? VERINA shows that 64 refinement rounds lift proof success only to about 20%, and Vero describes agents spending budget on stuck proofs rather than refactoring. That suggests extra rounds mostly buy more of the same behavior. It is still a real competitor, so every comparison is at matched total tokens, and the single-agent baseline gets best-of-n sampling with the arbiter's share of tokens reallocated to extra samples.
Why not just use a fixed "restart after k failures" rule? This is the most dangerous alternative because it is cheap and may capture most of the benefit. The stuck-checkpoint dataset settles it directly. If plan-level and local stalls are indistinguishable from goal summaries, a tuned $k$ will match the arbiter on recovered solves, and we will report that as the finding. Our hypothesis requires the arbiter to classify checkpoints better than chance and better than "always replan after $k$."
Why not just let P³'s single agent make the retreat decision, since it already distinguishes plan-level errors? P³ shows that the distinction matters but not who should make it. VERINA's finding that implementation detail degrades specification-level judgment is our reason to suspect self-assessment. The full-context arbiter ablation answers this cleanly. If seeing the code does not hurt the decision, P³'s single-agent design is sufficient and our restriction is unnecessary.
Why not just add an independent critic whose feedback is advisory? SGCR and the Criticize-Reflect loop in the organized-teams paper both use critics without veto power. Vero's observation that agents treat code as fixed scaffolding predicts that authors will ignore advice to replan, but this is untested. The advisory-arbiter ablation measures how often authors override a "replan" verdict and what that costs.
Proposed Experiments
The first and cheapest decisive check is offline. On the stuck-checkpoint dataset we compare restricted-view arbiters, full-context arbiters, the tuned $k$ rule, and the author's own P³-style self-classification. We report decision accuracy on unambiguous checkpoints and the fraction of oracle-recoverable solves each decider recovers. The proposal survives this stage only if the restricted arbiter beats both the full-context arbiter and the $k$ rule.
The end-to-end evaluation then runs all baselines on VERINA (full code-spec-proof track), Vero (both tracks), and Lean4Commit0. We report solve rate, cost per solve, wall-clock time, and the number of replans per solve. One diagnostic tests the editorial mechanism directly: proof-to-code ratio in solved runs. If arbiters work as intended, solved runs should show more code change after a verdict and less proof text spent on abandoned plans than Vero's single agents. A second diagnostic measures over-replanning: how often the arbiter rejects a plan whose repair branch would have succeeded. This is the bureaucratic drag a skeptic expects from hierarchy.
The result that would most clearly confirm the claim is a restricted arbiter that beats P³ and the full-context arbiter at equal budget on Vero full solves, with the gap concentrated on instances where the baseline stalls in proof repair. A null result in which the $k$ rule or the full-context arbiter matches it would also be informative. It would show that the useful part of "editorial structure" in this setting is a restart schedule, not a separate judge.
References
- VERINA: Benchmarking Verifiable Code Generation
- Vero: Can AI Agents Build Formally Verified Software Repositories?
- P$^{3}$: Joint Program-and-Proof Planning for Verified Code Generation
- IEEE Transactions on Cybernetics publication information
- Dilemmas and Governance Strategies in the Implementation of the Modern Apprenticeship System in Chinese Vocational Colleges: A Perspective from Principal-Agent Theory
- CooperBench: Benchmarking Cooperation in Coding Agents
- Embodied LLM Agents Learn to Cooperate in Organized Teams
- SGCR: A Specification-Grounded Framework for Trustworthy LLM Code Review