Cascaded Code Editing: Large-Small Model Collaboration for Effective and Efficient Code Editing

Chaozheng Wang, Zezhou Yang, Shuzheng Gao, Cuiyun Gao, Zongjie Li, Yichen Li, Ting Peng, Hailiang Huang, Yuetang Deng, Michael R. LyuarXiv 2026 · arXiv

TL;DR

Large language models (LLMs) excel at code editing but waste massive amounts of compute rewriting entire files of unchanged code. This paper introduces a cascaded framework that splits the task: a large model generates a concise "edit sketch" of the changes, and a specialized, fine-tuned small model merges it into the codebase. This collaboration not only slashes inference costs and time but surprisingly beats direct editing by large models alone.

What Problem Does This Paper Address?

Automated code editing requires an AI to understand both natural language instructions and complex codebases. Large frontier models (like DeepSeek-R1 or Claude-4) have the reasoning power to do this well, but they suffer from a massive efficiency bottleneck: they typically regenerate the entire file, even though empirical studies show that only about 15% of code actually changes during an update. This wastes tokens, time, and money.

Smaller models are faster and cheaper, but they lack the long-context reasoning capabilities to accurately locate and apply these edits without breaking things. The authors tackle this effectiveness-efficiency trade-off by figuring out how to successfully combine the reasoning power of large models with the speed of small models.

Methods

The authors propose a two-stage Cascaded Code Editing framework:

  1. Edit Sketch Generation: A powerful large model (e.g., DeepSeek-R1) acts as the planner, analyzing the code and instructions to generate a concise "sketch" containing only the modified code snippets.
  2. Edit Sketch Application: A smaller model (e.g., Qwen2.5-Coder) acts as the builder, mechanically merging the sketch into the original file.

To make small models competent at the application phase, the authors developed two progressive supervised fine-tuning (SFT) strategies:

  • Curriculum-based Long-Context SFT (CLC SFT): This trains the small model in stages—first on short-context instances for quick convergence, then on long-context data augmented with multi-file scenarios to simulate real-world editing.
  • Generalized Curriculum Long-Context SFT (G-CLC SFT): To prevent the small model from over-specializing and losing its general programming capabilities, this strategy mixes the sketch-application data with general-domain coding data (from the OpenCoder dataset) at a 1:1 ratio.

Datasets

  • Sketch Application Dataset (New): The authors curated the first large-scale dataset specifically for sketch application, containing 118,280 training instances (~800M tokens). It consists of 73,762 real-world GitHub commit-based records and 44,518 AI-synthesized examples.
  • Sketch Application Benchmark (New): A distinct evaluation benchmark consisting of 1,981 high-fidelity, human-evaluated samples to test how accurately models merge code.
  • Downstream Benchmarks: For end-to-end code editing evaluation, they utilized established benchmarks: Aider Polyglot (225 problems) and CanItEdit (210 problems).

Key Insights and Ideas

  • Bypassing the "Context Tax": Traditional scoped editing methods (like search-and-replace or unified diffs) suffer from a "Context Tax"—they must generate extensive surrounding code to anchor the edit location, which inflates token counts. The cascaded approach completely sidesteps this by letting the small model handle the localization mechanically.
  • Inverting the Performance Gap: Intuitively, a cascade should at best match the large model's accuracy while saving time. However, by offloading the tedious merging task, the large model avoids "attention drift" in long files, actually improving overall editing accuracy.
  • Small Models Need Generalization: Training small models only on sketch application degrades their performance on downstream tasks. Mixing in general coding data (G-CLC SFT) is crucial for real-world robustness.

Key Results

  • Superior Accuracy: On the Aider benchmark, pairing DeepSeek-R1 with a G-CLC SFT fine-tuned Qwen2.5-Coder 14B achieved a Pass@2 score of 75.1%, representing an 11.1% absolute improvement over direct editing with DeepSeek-R1 alone (67.6%).
  • Massive Resource Savings: For DeepSeek-R1, the cascade reduced execution time by 13% and costs by 19%. When paired with DeepSeek-V3, it cut total time by up to 36% and costs by 37%.
  • Small Model Boost: The specialized training strategies improved small models' performance on the sketch application benchmark by an average of 7.1 points on CodeBLEU and 40.9 points in Exact Match.
Method (Aider Benchmark) Pass@1 Pass@2 Total Time Cost ($)
DeepSeek-R1 (Direct) 35.6 67.6 27.31h 6.54
DeepSeek-R1 + QC 14B (Ours) 39.1 75.1 23.80h (↓13%) 5.32 (↓19%)

Limitations and Future Work

  • Model Scope: The evaluation was primarily limited to specific model families (DeepSeek for generation and Qwen2.5-Coder for application). The generalizability to other commercial or open-source models remains to be fully explored.
  • Real-World Latencies: The calculated time and cost savings are based on token throughputs and API pricing; real-world deployment benefits can fluctuate based on hardware configurations and network latencies.
  • Future Directions: The authors suggest extending this collaboration to multi-agent systems or iterative self-correction loops where the small model provides compilation feedback to the large model.

Summary updated

Abstract

Code editing constitutes a fundamental practice in software development, wherein developers modify existing codebases according to natural language requirements. Accurate code editing necessitates a comprehensive understanding of both the existing codebase and the modification requirements. Although large language models (LLMs) have demonstrated promising performance in code editing tasks, they suffer from substantial inefficiency by generating entire modified files that largely consist of unchanged code. While smaller models could potentially address this inefficiency, they typically lack the capacity to effectively comprehend long code contexts required for accurate editing. To ensure both effectiveness and efficiency, we propose to decompose code editing into a two-stage cascade: \textbf{edit sketch generation}, wherein a large model first produces concise sketches representing the requisite modifications (the more challenging phase), and \textbf{edit sketch application}, wherein a smaller model integrates these sketches into the original code to produce the final output edited code (the simpler phase). This cascaded design reduces the number of tokens generated by the large model, as the majority of the output is handled by the smaller, more efficient model, thereby enhancing overall efficiency. However, the effectiveness of this approach is constrained by current small models' limited capabilities in handling long-context scenarios and cross-file dependencies, which are essential for accurate sketch application in real-world codebases. To address these limitations and enhance smaller models' sketch application capabilities, ...
BibTeX
@misc{wang2026cascaded,
  author = {Chaozheng Wang and Zezhou Yang and Shuzheng Gao and Cuiyun Gao and Zongjie Li and Yichen Li and Ting Peng and Hailiang Huang and Yuetang Deng and Michael R. Lyu},
  title = {{Cascaded Code Editing: Large-Small Model Collaboration for Effective and Efficient Code Editing}},
  year = {2026},
  eprint = {2604.19201},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2604.19201},
}

Loading related content...

Figure