Program Repair via Sequential Edit Operations

40 concepts30 papers1 proposal

When a standard large language model (LLM) attempts to fix a bug or update software, its default behavior is to rewrite the entire source file or code block from scratch. This monolithic approach introduces significant inefficiencies: it wastes computational resources on reproducing code that is already correct, consumes massive context window capacity, and exposes the model to unnecessary failure modes, such as introducing accidental syntax or indentation errors in unrelated sections. As Jimenez et al. demonstrate in SWE-bench: Can Language Models Resolve Real-world Github Issues?, forcing models to regenerate full files to resolve real-world software issues results in a massive performance gap compared to targeted engineering.

To overcome this bottleneck, a powerful paradigm shift reframes code generation and repair not as monolithic synthesis, but as sequential editing. Instead of predicting raw code tokens from scratch, models are trained to predict a sequence of discrete edit operations—such as insertions, deletions, and replacements—relative to an existing "candidate" program.

Shifting the Generative Focus

The core intuition behind edit-based program repair is simple: focus the model's generative capacity exclusively on the delta between the buggy code and the correct code.

By treating the existing buggy code as a structural hint rather than starting from scratch, the model drastically reduces its search space. As Hu et al. show in Fix Bugs with Transformer through a Neural-Symbolic Edit Grammar, predicting minimal edit operations prevents the model from over-relying on simply copying the input, forcing it to localize and resolve the semantic error directly.

Furthermore, this formulation preserves the developer's original intent and logic. In educational contexts, Koutcheme et al. note in Using Program Repair as a Proxy for Language Models' Feedback Ability in Programming Education that constraining program repair to minimal changes ensures the system respects a student's initial logic rather than completely rewriting the program.

Why it Works: Mathematical and Structural Tractability

The primary reason edit-based models outperform full-text generators lies in probability distribution modeling. Modeling the probability of a small set of edit operations conditioned on the surrounding context is mathematically more tractable than modeling the joint probability of an entire new program.

Monolithic Generation:
[Buggy Input] ---> LLM ---> [Generates Entire New File Token-by-Token] (High risk of syntax drift)

Edit-Based Generation:
[Buggy Input] ---> LLM ---> [Action: Insert ";" at Line 4, Col 12] ---> [Correct Output] (Surgical precision)

Alon et al. formalize this concept in A structural model for contextual code changes, demonstrating that predicting structural paths or edits within Abstract Syntax Trees (ASTs) yields substantial accuracy gains over full-text generation.

This formulation also allows models to leverage optimal paths derived from classical algorithms. Instead of guessing arbitrary token sequences, the model learns to navigate a mathematically optimal path:

Granularity of Edits

Edit-based approaches operate at various levels of granularity depending on the nature of the task:

  1. Line-Level and Search-and-Replace Blocks: Early systems like DeepFix by Gupta et al. output specific line indices and their corrected contents. Modern iterations, such as the framework proposed by Peng et al. in SoRFT: Issue Resolving with Subtask-oriented Reinforced Fine-Tuning, use localized search-and-replace blocks to minimize the output space and ensure patch reliability in large codebases.
  2. Diffs and Patches: Generating standard unified diffs reduces context window consumption and prevents formatting errors, as highlighted by Aggarwal et al. in DARS: Dynamic Action Re-Sampling to Enhance Coding Agent Performance by Adaptive Tree Traversal.
  3. Token-Level Actions: For precise syntax correction (e.g., a missing brace or semicolon), token-level editing prevents the corruption of surrounding code logic, a mechanism leveraged by Gupta et al. in Deep Reinforcement Learning for Syntactic Error Repair in Student Programs..

Regardless of the granularity, localizing the initial point of failure allows models to maintain correct prefixes while focusing computational resources strictly on specific erroneous segments, a benefit thoroughly explored by Jiang et al. in LATTE: Improving Latex Recognition for Tables and Formulae with Iterative Refinement.

Mimicking Human Workflows and Data Efficiency

Beyond computational savings, sequential editing aligns closely with how humans write and debug software: incrementally, one chunk at a time.

In Training Language Models on Synthetic Edit Sequences Improves Code Synthesis, Piterbarg demonstrates that refactoring static code into synthetic sequences of incremental insertions significantly improves code synthesis performance and test-time compute scaling across various model sizes. By learning to predict a series of logical edits rather than a single monolithic block, the model mimics the incremental construction process of human programming.

This alignment also yields massive data efficiency. Jones et al. show in Learning to Edit Visual Programs with Self-Supervision that learning local modifications allows systems to achieve comparable performance levels with up to an order of magnitude less training data than learning to generate programs from scratch.

To enforce this precision, models can be pretrained on explicit "edit plans." As developed by Zhang et al. in CoditT5: Pretraining for Source Code and Natural Language Editing, forcing a model to first generate an explicit plan of insertions and deletions before producing the final code prevents the introduction of sweeping, irrelevant changes.

Finally, this incremental view enables process-oriented training. As shown by Liang et al. in Condor: A Code Discriminator Integrating General Semantics with Code Details, decomposing code fixes into discrete hunks allows models to learn from intermediate, transitional states, training the system to identify the exact incremental changes required to transform a buggy implementation into a correct one.

Key Foundational Work

Automated Semantic Grading of Programs (2012) — contributed 2 concept elements to this direction.
DeepFix: Fixing Common C Language Errors by Deep Learning (2017) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_edit_based_program_repair_f7c2063d,
  title = {edit-based program repair},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/edit-based-program-repair/rdr_f7c2063d54ca4fa889ba4315d3cf923c},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_edit_based_program_repair_f7c2063d_v1,
  title = {edit-based program repair},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/edit-based-program-repair/rdr_f7c2063d54ca4fa889ba4315d3cf923c/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake