Large Language Models (LLMs) have become remarkably proficient at "greenfield" development—generating entire functions or scripts from a blank file. However, professional software engineering rarely starts from scratch. Instead, it involves an iterative cycle of debugging, refactoring, and adapting existing codebases.
Recent research reveals a significant "editing gap": models that excel at code synthesis often stumble when asked to modify existing code. As Aggarwal et al. and Song et al. observe, this discrepancy is a major hurdle for AI-powered development, as the majority of a developer's time is spent on maintenance rather than initial creation.
The Taxonomy of Code Modification
To understand why models struggle, we must first define what "editing" entails. Guo et al. propose a structured framework in CodeEditorBench, categorizing editing into four distinct scenarios:
- Code Debug: Identifying and fixing errors.
- Code Translate: Converting code between programming languages.
- Code Polish: Improving efficiency or readability without changing functionality.
- Code Requirement Switch: Adapting code to new or modified logic.
Among these, the authors found that Requirement Switching is the most difficult. It requires the model to perform significant logical restructuring, a task where even state-of-the-art models show their lowest success rates. This difficulty is compounded when instructions are "lazy" or minimal, forcing the model to infer intent from context—a challenge highlighted by Cassano et al. in their CANITEDIT benchmark.
Why Editing is Harder than Synthesis
The difficulty of editing stems from the need to balance new instructions against existing constraints. In synthesis, the model has total freedom. In editing, the model must respect the existing architecture, variable naming, and logic while injecting changes.
This challenge is particularly visible in specialized domains. For instance, Tarasova et al. introduced SWE-InfraBench to evaluate models on Infrastructure-as-Code (IaC). They found that models struggle with the incremental modifications required for cloud infrastructure, where a small change can have cascading effects across a complex system.
Furthermore, Soni et al. argue that the bottleneck in automated software engineering tasks—like issue reproduction—is not finding where the bug is (localization), but rather the act of editing the code to create a failing test case. Even when a model knows exactly which file to change, it often fails to produce a valid, functional modification.
Strategies for Robust Adaptation
To close this gap, researchers are moving beyond general pre-training toward specialized editing frameworks.
Instruction Tuning for Edits
The first step is providing models with the right data. Song et al. developed InstructCoder, the first large-scale dataset specifically designed for code editing. By fine-tuning models on over 114,000 diverse editing tasks, the authors demonstrated that open-source models could reach the performance levels of proprietary models like ChatGPT on refactoring tasks.
Selective Knowledge Transfer
A risk of specialized tuning is "catastrophic forgetting," where a model becomes good at editing but loses its general reasoning abilities. To combat this, Aggarwal et al. introduced NextCoder, which uses a "Selective Knowledge Transfer" (SeleKT) algorithm. This approach ensures the model learns the nuances of bug fixing and optimization without sacrificing its foundational programming knowledge.
New Pre-training Objectives
Some researchers suggest that the standard "predict the next token" objective is insufficient for editing. Zhu et al. propose DivoT5, a framework that treats code editing as a "directional diffusion" process. Instead of a single-step generation, the model learns to iteratively refine code toward a target state.
While these methods improve fine-grained modifications, Fan et al. note that high-level structural refactoring remains an open challenge. Future progress likely depends on teaching models to understand not just the syntax of a change, but the architectural intent behind it.
Key Foundational Work
CodeEditorBench: Evaluating Code Editing Capability of LLMs (2025) — contributed 6 concept elements to this direction.
Can It Edit? Evaluating the Ability of Large Language Models to Follow Code Editing Instructions (2024) — contributed 3 concept elements to this direction.
Exploring the Capabilities of LLMs for Code-Change-Related Tasks (2025) — contributed 2 concept elements to this direction.