Early neural automated program repair (APR) systems treated bug fixing as a sequence-to-sequence translation task, mapping a buggy code snippet directly to a corrected version. As shown by Berkay Berabi in TFix: Learning to Fix Coding Errors with a Text-to-Text Transformer, deep learning models can learn complex, human-like edits from massive commit histories. However, this translation paradigm requires extensive supervised training data and struggles to generalize when the search space of potential fixes becomes infinitely large.
Recent research has shifted toward leveraging Large Language Models (LLMs) through targeted prompt engineering and context augmentation. Instead of retraining models to learn specific bug-fix transformations, this direction focuses on providing the LLM with the exact syntactic, semantic, and diagnostic context it needs to deduce the correct code.
Framing Repair as In-Context Completion
One of the core insights in modern program repair is that an LLM does not always need to generate a patch from scratch; instead, it can fill in the blanks. By transforming code repair into a "cloze-style" or mask-prediction task, the surrounding codebase acts as a natural constraint on the model's generation.
Chunqiu Steven Xia et al. demonstrated this intuition in Less Training, More Repairing Please: Revisiting Automated Program Repair via Zero-shot Learning. By masking out a buggy line and allowing a pre-trained model to predict the missing tokens in a zero-shot setting, the model leverages its bidirectional understanding of code syntax. This bidirectional context ensures that the generated patch naturally fits the surrounding logic, bypassing the need for noisy historical training data.
To preserve the developer's original intent during this completion process, Chunrong Fang et al. introduced a hybrid approach in GAMMA: Revisiting Template-based Automated Program Repair via Mask Prediction. They inserted the original erroneous line as a comment within the prompt. This comment serves as a semantic hint, grounding the model's mask-prediction process and preventing it from drifting away from the program's intended functionality.
Augmenting Prompts with Diagnostic and Structural Context
An isolated buggy function rarely contains enough information for an LLM to diagnose why a failure occurred. To bridge this gap, context-aware program repair systems inject external diagnostic feedback and structural code hierarchies directly into the prompt.
+-------------------------------------------------------------+
| Prompt Context |
| 1. Diagnostic Feedback (e.g., Compiler Errors, Test Cases) |
| 2. Structural Context (e.g., Variable Declarations) |
| 3. Scope Constraints (e.g., Entire Function Bodies) |
+-------------------------------------------------------------+
|
v
+-------------------------------------------------------------+
| Large Language Model |
+-------------------------------------------------------------+
|
v
+-------------------------------------------------------------+
| Syntactically & Semantically Correct Patch |
+-------------------------------------------------------------+
Providing explicit error context allows general-purpose models to excel without language-specific engineering. For example, Harshit Joshi et al. showed in Repair Is Nearly Generation: Multilingual Program Repair with LLMs that appending compiler error messages directly to the prompt allows a single LLM to resolve bugs across multiple programming languages, matching or exceeding specialized symbolic tools.
However, compiler errors often point to symptoms rather than root causes, especially when a bug depends on code defined elsewhere. To resolve these non-local dependencies, Xueyang Li et al. developed TransRepair: Context-aware Program Repair for Compilation Errors, which automatically extracts variable declaration contexts from distant files and appends them to the prompt. Similarly, Bowen Zhou et al. observed in Dolphin: Moving Towards Closed-loop Auto-research through Thinking, Practice, and Feedback that providing local code structures and function hierarchies alongside error logs dramatically improves debugging success rates compared to providing raw error messages alone.
Exploiting Redundancy via Retrieval-Augmented Repair
Software repositories exhibit a high degree of structural and logical redundancy. Developers frequently make similar mistakes, and valid solutions to a given problem often share algorithmic patterns. Retrieval-Augmented Generation (RAG) exploits this property by finding relevant code examples and inserting them into the LLM's prompt as in-context demonstrations.
The utility of this approach was demonstrated by Matthew Jin et al. in InferFix: End-to-End Program Repair with LLMs and Weishi Wang et al. in RAP-Gen: Retrieval-Augmented Patch Generation with CodeT5 for Automatic Program Repair. By retrieving semantically similar historical bug-fix pairs from a codebase, the system provides the LLM with a concrete blueprint for the repair. This allows the model to distill specific, localized repair logic that would be difficult to generalize from its static pre-training data alone.
This reference-based prompting is equally powerful when fixing complex logic errors in educational settings. In Peer-aided Repairer: Empowering Large Language Models to Repair Advanced Student Assignments, Qianhui Zhao et al. demonstrated that retrieving a correct "peer solution"—a working assignment written by another student solving the same problem—provides the LLM with a functional roadmap. This structural reference guides the model to correct intricate logic bugs while respecting the original author's implementation style.
Designing Prompts Around Model Limitations
Effective context engineering also requires mitigating the inherent structural weaknesses of LLMs, such as their poor spatial reasoning when generating code modifications.
When LLMs are asked to generate standard patch files (such as unified diffs), they frequently hallucinate incorrect line numbers, causing otherwise perfect patches to fail during deployment. To circumvent this limitation, Kareem Shehada et al. proposed a structural shift in Rethinking Kernel Program Repair: Benchmarking and Enhancing LLMs with RGym. Instead of forcing the model to pinpoint precise line edits, the repair prompt instructs the model to rewrite the entire function body. Replacing code at the function level preserves structural integrity, eliminates line-number hallucinations, and simplifies the automated integration of patches into the codebase.
Key Foundational Work
Less Training, More Repairing Please: Revisiting Automated Program Repair via Zero-shot Learning (2022) — contributed 3 concept elements to this direction.