To Diff or Not to Diff? Structure-Aware and Adaptive Output Formats for Efficient LLM-based Code Editing

Wei Cheng, Yongchang Cao, Chen Shen, Binhua Li, Jue Chen, Yongbin Li, Wei Hu

TL;DR

This paper introduces BLOCKDIFF and FUNCDIFF, new structure-aware diff formats that allow LLMs to edit code by rewriting syntactically coherent blocks rather than arbitrary line fragments. To maximize efficiency, the authors also propose ADAEDIT, an adaptive strategy that trains models to dynamically choose the most token-efficient output format (diff vs. full code), resulting in over 30% reduction in latency and cost for long-code editing without sacrificing accuracy.

What Problem Does This Paper Address?

When you ask an AI coding assistant to fix a bug, it usually generates the entire file from scratch, even if it's only changing one line. This "full-code generation" paradigm is reliable but incredibly wasteful, leading to high latency and high API costs.

The obvious alternative is to use "diff" formats (like those used by Git). However, standard diffs are problematic for LLMs:

  1. Fragile Offsets: Formats that rely on line numbers (like UNIDIFF) are hard for probabilistic models to generate accurately. One wrong number breaks the whole patch.
  2. Fragmented Hunks: Content-addressed diffs often produce "fragmented hunks"—disjointed snippets of code that break syntactic integrity. Since LLMs are pre-trained on well-formed, complete code, generating these fragments feels "unnatural" to them, leading to lower accuracy.
Examples of conventional diff formats, showing line-number fragility and fragmented hunks.

Methods

The authors propose a two-pronged solution: better formats and an adaptive selection strategy.

1. Structure-Aware Diff Formats

Instead of arbitrary lines, the authors use the Abstract Syntax Tree (AST) to define "logical modules" for editing:

  • BLOCKDIFF: Edits occur at the level of fine-grained AST nodes, such as control structures (loops, branches) and functions.
  • FUNCDIFF: A coarser version that ignores internal control structures to favor broader structural stability by rewriting entire functions.

By aligning diff hunks with these syntactically coherent units, the LLM generates code that looks more like the data it was trained on, restoring the "naturalness" of the generation process.

2. ADAEDIT: The Adaptive Strategy

No single format is always best. If a change is pervasive (e.g., renaming a variable used on every line), a diff might actually use more tokens than just outputting the full code.

ADAEDIT is a training strategy where the model internalizes this cost-benefit logic. During Supervised Fine-Tuning (SFT), the ground truth for each sample is set to whichever format—diff or full code—is shorter in terms of tokens. At inference time, the model implicitly predicts and uses the most efficient format for the specific task.

Overview of structure-aware formats and the ADAEDIT selection logic.

Datasets

The researchers utilized a variety of datasets to ensure their findings were robust across different tasks and languages:

  • Training: OCEData (synthetic Python), InstructCoder (Python snippets), and the JavaScript subset of CommitPackFT.
  • Evaluation:
    • EditEval & CanItEdit: Instruction-guided editing benchmarks.
  • HumanEvalFix: A bug-fixing benchmark based on unit tests.
  • Aider: Simulates a realistic two-stage coding workflow (initial generation followed by test-driven repair).

Key Insights and Ideas

  • Naturalness = Accuracy: The failure of standard diffs isn't just about math (line numbers); it's about syntax. LLMs struggle to generate "broken" snippets. Structure-aware formats solve this by ensuring every hunk is a valid code block.
  • The Efficiency Threshold: There is a clear crossover point where diffs become less efficient than full code. ADAEDIT successfully identifies this threshold, achieving a selection accuracy of over 90%.
  • Usability Matters: Beyond just getting the code right, the authors measured "usability" via linter checks and patch-apply success rates. Structure-aware formats significantly outperformed line-indexed formats, which often produced code that failed basic syntax checks.
Usability comparison showing that structure-aware formats (FUNCDIFF/BLOCKDIFF) maintain high linter and patch success rates.

Key Results

  • Cost & Latency: On long-code editing tasks (input > 500 tokens), ADAEDIT reduced total output tokens and latency by over 30% compared to full-code generation.
  • Accuracy Parity: Despite the massive token savings, structure-aware formats matched or even slightly exceeded the accuracy (Pass@1) of the full-code baseline across multiple models (Qwen2.5-Coder, DeepSeek-Coder).
  • Scaling Efficiency: As the input code length grows, the cost of full-code generation scales linearly, while ADAEDIT remains relatively flat, making it ideal for large-scale repository editing.
Comparison of output tokens across different code scales, highlighting ADAEDIT's efficiency for long files.

Limitations and Future Work

  • Base Model Dependency: The benefits of structural formats are more pronounced in more capable models. Smaller or weaker models may still struggle with the logic required to generate even structured diffs.
  • Data Scarcity: There is a general lack of high-quality, large-scale training data for complex, repository-level code editing.
  • Future Directions: The authors suggest exploring Reinforcement Learning (RL) to allow models to autonomously discover even more optimized editing formats by using token efficiency and functional correctness as reward signals. They also plan to investigate "fluid granularity" that isn't strictly tied to fixed AST boundaries.

Summary updated

Abstract

Large Language Models (LLMs) are increasingly used for code editing, yet the prevalent full-code generation paradigm suffers from severe efficiency bottlenecks, posing challenges for interactive coding assistants that demand low latency and cost. Despite the predominant focus on scaling model capabilities, the edit format itself has been largely overlooked in model training. In this paper, we begin with a systematic study of conventional diff formats and reveal that fragile offsets and fragmented hunks make generation highly unnatural for LLMs. To address it, we introduce BlockDiff and FuncDiff, two structure-aware diff formats that represent changes as block-level rewrites of syntactically coherent units such as control structures and functions. Furthermore, we propose AdaEdit, a general adaptive edit strategy that trains LLMs to dynamically choose the most token-efficient format between a given diff format and full code. Extensive experiments demonstrate that AdaEdit paired with structure-aware diff formats consistently matches the accuracy of full-code generation, while reducing both latency and cost by over 30% on long-code editing tasks.
BibTeX
@inproceedings{cheng2026diff,
  author = {Wei Cheng and Yongchang Cao and Chen Shen and Binhua Li and Jue Chen and Yongbin Li and Wei Hu},
  title = {{To Diff or Not to Diff? Structure-Aware and Adaptive Output Formats for Efficient LLM-based Code Editing}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1483},
}

Loading related content...

Figure