Abstract
We address the problem of predicting edit completions based on a learned model that was trained on past edits. Given a code snippet that is partially edited, our goal is to predict a completion of the edit for the rest of the snippet. We refer to this task as the EditCompletion task and present a novel approach for tackling it. The main idea is to directly represent structural edits. This allows us to model the likelihood of the edit itself, rather than learning the likelihood of the edited code. We represent an edit operation as a path in the program's Abstract Syntax Tree (AST), originating from the source of the edit to the target of the edit. Using this representation, we present a powerful and lightweight neural model for the EditCompletion task. We conduct a thorough evaluation, comparing our approach to a variety of representation and modeling approaches that are driven by multiple strong models such as LSTMs, Transformers, and neural CRFs. Our experiments show that our model achieves a 28% relative gain over state-of-the-art sequential models and 2x higher accuracy than syntactic models that learn to generate the edited code, as opposed to modeling the edits directly.
Introduction
Software development is an inherently iterative process where a single modification often triggers a cascade of necessary changes across a codebase. For instance, updating a method's signature or changing a conditional check frequently requires corresponding adjustments in variable assignments or return statements within the same scope. While traditional code completion tools excel at predicting the next few tokens of new code, they are often ill-equipped to handle the "Edit Completion" task: predicting the remaining parts of a transformation based on a partial edit.
Existing approaches typically treat code as flat text, ignoring the underlying hierarchical structure of programming languages. Furthermore, many models attempt to predict the entire resulting code block ($Pr(Y | X)$), which places an unnecessary burden on the model to learn how to write valid code from scratch. This paper introduces $C^3PO$, a model that shifts the focus to modeling the edits themselves ($Pr(\Delta_P | \Delta_C)$), representing them as structural paths within an Abstract Syntax Tree (AST).
Structural Edit Representation
The core innovation of this work is the representation of edits as AST paths. Instead of viewing a change as a textual diff, the model identifies the specific nodes in the AST that are affected. An edit operation—such as moving, deleting, updating, or inserting a node—is represented as a path from a source node to a target node within the tree.
This structural approach allows the model to generalize patterns across different variable names and specific logic. For example, it can learn the abstract concept of "swapping the order of two arguments in a method call" by recognizing the path-based relationship between the nodes, regardless of the specific identifiers involved.
The $C^3PO$ Architecture
$C^3PO$ is designed to be both powerful and lightweight. The architecture consists of:
- Encoder: An LSTM-based component that encodes the sequence of edits that have already occurred in the surrounding context.
- Decoder: A Pointer Network that, instead of selecting tokens from a fixed vocabulary, "points" to valid paths in the AST to construct the predicted edit script.
By focusing on the delta rather than the full code, $C^3PO$ remains remarkably efficient. The model utilizes only 750,000 parameters, a stark contrast to the 10M to 45M parameters found in the Transformer-based baselines it eventually outperformed.
Experimental Evaluation
The researchers evaluated $C^3PO$ using a new large-scale dataset derived from over 300,000 commits across 53 popular C# repositories on GitHub. The final dataset consisted of approximately 50,000 high-quality examples of contextual edits.
Key Results
- Accuracy: $C^3PO$ achieved an exact-match accuracy of 53.2%.
- Relative Gains: The model demonstrated a 28% relative gain over state-of-the-art sequential models like LaserTagger.
- Comparison to Syntactic Models: It achieved 2x higher accuracy than Path2Tree, a model that attempts to generate the final edited code rather than the edit script itself.
- Efficiency: Training was completed in just 9 hours, compared to the 20-24 hours required by larger competitive models.
Conclusion
By representing edits as structural paths in an AST, $C^3PO$ demonstrates that modeling the "likelihood of the edit" is a more effective strategy for edit completion than modeling the "likelihood of the code." This lightweight, high-accuracy approach paves the way for more intelligent IDE assistants and automated code review tools that can suggest complex refactorings with minimal computational overhead.
Summary updated