When a language-model agent fixes a bug in a real software repository, it has to decide where to edit before it decides what to write. The choice of which files, classes and functions to touch fails in two opposite ways. Agents over-edit: they change code outside the region the fix needs and break things downstream. They also under-edit: they change a function's signature or data representation and leave its callers alone, which produces what the authors of $A_2$DEPT call the "partial migration" trap. Both failures break a logical rule about scope. If a function needs editing, so does the file that contains it. If a function's interface changes, every caller that depends on that interface needs editing too. Today's agents enforce these rules only by asking the model in natural language to respect them.
This proposal treats edit-scope prediction as hierarchical multi-label classification. The labels are the repository's own functions, classes and files, linked by "this implies that" edges, and the prediction is made coherent with those edges by construction. We reuse a layer from hierarchical classification that guarantees a parent's score is never lower than any of its descendants' scores Coherent Hierarchical Multi-Label Classification Networks. We apply it to a graph that combines code containment with caller dependencies. The resulting scope model then runs inside the repair loop. After each candidate patch, it flags implied edits the patch skipped and edits it made with little support. Those flags become structured feedback in place of another unguided retry.
Core Claim
A small, trained scope predictor whose outputs cannot violate scope implications will raise issue-resolution rates on multi-file repository bugs. It will do this mainly by reducing partial-migration and over-edit failures, and it will do it better than three alternatives at matched compute: a flat predictor followed by post-hoc reconciliation, a purely static caller closure, and prompting the agent to "check callers." The key ingredient is coherence by construction on a noisy implication graph built automatically for each repository. The hierarchical-classification literature has not tested this setting, because its methods assume a fixed, clean taxonomy known in advance C-HMCNN.
Where This Sits in Prior Work
Two kinds of evidence motivate the proposal.
The first is direct evidence that scope errors drive failures. $A_2$DEPT evolves full solver programs with a tree of edit operators and a self-healing loop that writes missing function definitions. Its authors report that this loop does not repair cross-function type mismatches after a leaf-level representation change. Adversarial Review uses a Reviewer and a Critic who must disagree in a structured way before any code is changed. Its authors report scope creep, meaning edits outside the intended region that break dependencies. MAGIS splits repair across retrieval, manager, developer and QA agents. Cross-file consistency there is negotiated only in a natural-language "kick-off meeting," and success falls as the number of touched files grows. All three handle scope in conversation or after the fact. We replace that with a coherence guarantee in the scope model's forward pass.
The second kind is evidence that unstructured iteration is weak. Demystifying GPT Self-Repair finds that GPT-4 self-repair beats independent resampling by only about 5–8% on APPS at matched compute, and that weaker models do worse than resampling. It also finds that human feedback repairs 52.6% of programs against 33.3% for self-generated feedback. Better-targeted feedback, not more rounds of it, is therefore the lever. Our scope flags are a cheap, automatic form of targeted feedback.
Three methods from hierarchical classification supply the technique. The Max Constraint Module and its loss from C-HMCNN give coherence with no extra parameters. The loss sends a parent's gradient to its most confident descendant. That paper's ablation also showed the module helps only when it is paired with this loss, not when it is applied at inference alone, and that result shapes our training design. Marginalization classifiers Dhall et al. offer an alternative way to compute coherent parent scores. We do not adopt them, because edit labels are not mutually exclusive children whose probabilities sum to the parent's.
Two results constrain the design. Coarse-to-fine top-down decoding Chen et al. suffers from error propagation: rejecting a parent wrongly prunes its whole subtree. File-then-function localization, which repository agents commonly use, has the same structure. We therefore compute coherence bottom-up and never prune. Post-hoc reconciliation Consistent probabilistic outputs for protein function prediction found that many heuristics lowered precision, while isotonic regression and KL projection helped. That makes isotonic reconciliation the honest flat-plus-fix-up baseline to beat rather than a strawman.
Research Program
For each repository snapshot we build an implication graph over candidate nodes, which are the files, classes and functions retrieved for the issue by BM25, following MAGIS's retrieval agent. Each function $f$ gets two binary labels: $e(f)$, "$f$ is edited," and $i(f)$, "$f$'s interface or the data contract it exposes changes." Edges encode implications. An interface change implies an edit, so $i(f) \Rightarrow e(f)$. Editing a member implies editing its container, so $e(f) \Rightarrow e(\text{class})$ and $e(\text{class}) \Rightarrow e(\text{file})$. An interface change implies edits to its callers, so $i(f) \Rightarrow e(g)$ for each statically resolved caller $g$. Python call graphs are approximate, so the graph is noisy by design. That noise is the regime C-HMCNN leaves open.
A base scorer $h$ assigns each label $v$ a raw probability $h(v)$. We use a cross-encoder over the issue text and the node's code, initialized from a code model. The coherent score is
$$s(v) = \max\Big(h(v),\ \max_{u \in \mathcal{D}(v)} h(u)\Big),$$
where $\mathcal{D}(v)$ is the set of labels that imply $v$ through any chain of edges. Because of this formula, $s$ can never assign a caller a lower score than an interface change it depends on. Training uses the max-constraint loss on SWE-bench-train, whose 19,000 instances span 37 repositories SWE-bench. Gold labels come from the reference patches. Interface-change labels come from automatic signature and return-type diffs. A backward-compatible change, such as adding a default argument, is labeled as no interface change. This is where the learned model can beat blind static closure.
Inside the repair loop, the agent first receives the nodes whose $s(v)$ clears a threshold as its working scope. After each candidate patch we re-score using the patch itself. A detector labels which interfaces the patch actually changed, and the coherent scores are recomputed. Two kinds of flag go back to the agent as feedback. "Missing implied edit" fires when the patch changed $f$'s interface but left a caller untouched. "Unsupported edit" fires when the patch touched a node with low $s$. Patches that add unsupported edits and fail more tests are rolled back to the last stable checkpoint, in the style of MEIC. Test execution remains the final judge. The scope model only decides where effort goes.
Baselines
The naive reference point is the same backbone language model run as a plain agent: retrieve, edit, run tests, retry, with the same iteration and token budget. Alongside it we run independent resampling at matched token count, since Demystifying GPT Self-Repair shows resampling is often as good as repair.
The strongest published systems in the file are MAGIS, at a 25.33% resolution rate on SWE-bench Lite with GPT-4 (MAGIS, which also reports SWE-Agent at 18.00% and AutoCodeRover at 16.11%), and Adversarial Review, at a 75.2% pass rate on SWE-bench Verified (Adversarial Review). These numbers come from different backbones and budgets, so we will not compare against them as reported. Instead we reimplement the Adversarial Review triad and MAGIS's role structure on our backbone, with identical retrieval, token budget and test access. We then add our scope module to each one. That pairing tests whether coherent scoping is complementary to structured review rather than a substitute for it. Beating a baseline means a statistically significant gain in resolution rate on the same instances at equal or lower tokens, measured by a paired bootstrap over issues.
Three baselines isolate the key ingredient. The first is the same base scorer $h$ trained with plain per-label binary cross-entropy and no coherence. The second is that flat scorer followed by isotonic reconciliation onto the implication graph at inference, the method Consistent probabilistic outputs found most reliable. The third is the coherence module applied at inference only, without the max-constraint loss, which mirrors the C-HMCNN ablation. The full method must beat all three on scope quality and on resolution rate to support the claim that coherence during training matters.
Why not just…?
Why not just run a static call-graph closure: whenever a patch changes a signature, add every caller to the scope? This is the most serious threat. For codebases with precise static types it may be nearly sufficient. In Python, static caller resolution is incomplete, so closure both misses dynamic callers and floods the scope with irrelevant ones. Closure also cannot tell a breaking change from a compatible one. We include closure as a baseline and report over-edit rate and resolution rate for it. If closure matches our method on resolution rate, the learned coherent model is not justified, and we will say so.
Why not just prompt the agent to check callers and containing classes before finishing? The file contains evidence against prompting-only structure. CodeTransOcean reports that chain-of-thought prompting degraded translation by pushing models into line-by-line work that lost track of module-level dependencies. Adversarial Review needed rigid disagreement constraints to escape false consensus. Neither result is about scope instructions specifically, so we run this prompt variant at matched tokens rather than assume it fails.
Why not just sample more patches and keep one that passes tests? On repository-level tasks, raw sampling was near zero in the original SWE-bench evaluation: Claude 2 reached 1.96%, and GPT-4 solved none of the sparse-retrieval subset SWE-bench. Resampling at matched compute is nonetheless a required baseline, because Demystifying GPT Self-Repair shows it often beats repair loops on smaller tasks.
Why not just fine-tune the agent on gold patches so it learns scope implicitly? That is a much more expensive intervention on the backbone. Our scorer is small, model-agnostic and auditable. As a partial check, we report whether scope gains persist when the backbone is swapped for a stronger model.
Proposed Experiments and Decisive Checks
We train on SWE-bench-train and evaluate on SWE-bench Lite and SWE-bench Verified, excluding any repositories that appear in training.
Scope quality is measured as label-level F1 against gold-patch nodes at each level (function, class, file). Coherence is measured as the rate of violations, defined as interface changes with unedited callers, in final patches. This mirrors the roughly 11% parent-child inconsistency rate measured for unreconciled protein classifiers in Consistent probabilistic outputs; no reported result was found on Lacuna for this violation rate in code agents, so we establish it. End-to-end metrics are resolution rate, over-edit rate (edited nodes absent from the gold patch) and tokens per resolved issue.
The decisive check is stratification by the number of files in the gold patch. If coherent scoping matters, gains over the flat-plus-isotonic baseline and the static-closure baseline should be near zero on single-function fixes and should grow with file count. That growth is the pattern MAGIS's complexity penalty predicts we can recover. A second check perturbs the implication graph by randomly dropping or adding a fraction of caller edges. This measures how quickly the benefit of coherence degrades with graph noise and tests the fixed-taxonomy assumption C-HMCNN leaves open. A third check counts partial-migration failures directly, as test failures traceable to an unupdated caller of a changed interface, in each condition. If coherence does not reduce this count, the proposed mechanism is not what drives any resolution gains, whatever those gains turn out to be.
References
- $A_2$DEPT: Large Language Model–Driven Automated Algorithm Design via Evolutionary Program Trees
- Coherent Hierarchical Multi-Label Classification Networks
- Adversarial Review: Structured Disagreement for Grounded Agentic Code Review
- MAGIS: LLM-Based Multi-Agent Framework for GitHub Issue Resolution
- Demystifying GPT Self-Repair for Code Generation
- Hierarchical Image Classification using Entailment Cone Embeddings
- Hierarchical Entity Typing via Multi-level Learning to Rank
- Consistent probabilistic outputs for protein function prediction
- SWE-bench: Can Language Models Resolve Real-world Github Issues?
- MEIC: Re-thinking RTL Debug Automation using LLMs
- CodeTransOcean: A Comprehensive Multilingual Benchmark for Code Translation