A Systematic Literature Review on Neural Code Translation

Xiang Chen, Jiacheng Xue, Xiaofei Xie, Caokai Liang, Xiaolin JuarXiv 2025 · DOI · arXiv

Abstract

This paper presents a systematic literature review (SLR) of 57 primary studies on neural code translation published between 2020 and 2025. It tracks the field's transition from specialized neural models to Large Language Models (LLMs), analyzing how researchers address challenges in syntax, semantics, and project-level context. The review synthesizes findings on data preprocessing, code representation, model construction, and evaluation metrics, highlighting the shift toward functional validation and prompt engineering.

Introduction

In the modern "polyglot" software landscape, developers frequently encounter the need to migrate legacy systems or maintain feature parity across multiple platforms. Manual code translation—such as migrating a codebase from COBOL to Java or Python to C++—is notoriously expensive, time-consuming, and error-prone. While Neural Machine Translation (NMT) has achieved significant success in natural languages, source code presents unique structural and semantic constraints.

Example of translating a factorial function from Java to Python, illustrating the shift in syntax and paradigms.

Code translation must respect strict syntax where a single character error can prevent compilation. Furthermore, semantic gaps between languages (e.g., dynamic vs. static typing) and the scarcity of project-level context make automated translation a complex frontier in software engineering. This review synthesizes five years of research to identify effective techniques and remaining challenges.

Methodology

The authors conducted a rigorous SLR following the Kitchenham guidelines. The process involved an automated search across major digital libraries to identify relevant literature, which was then filtered down to 57 high-quality primary studies.

Major digital libraries and search engines utilized for the automated search phase of the SLR.

The studies were analyzed through seven distinct lenses: task characteristics, data preprocessing, code modeling (textual vs. structural), model construction (training vs. prompting), post-processing, and evaluation methodologies.

Dataset Landscape

The review identifies several "gold standard" benchmarks that have shaped the community's progress:

  • TransCoder: The most widely used benchmark (appearing in 15 studies), leveraging large-scale data from GeeksforGeeks.
  • CodeXGLUE: A versatile suite supporting multiple languages and tasks.
  • Avatar & CodeNet: Large-scale datasets frequently used for Java-to-Python translations and competitive programming.
  • Chen’s Dataset: A foundational resource for Java-C# migration.

Recent trends show a shift toward AVATAR-TC and PolyHumanEval, which prioritize compilability and functional validation over simple text similarity.

Evolution of Techniques and Insights

The "Text vs. Structure" Debate

Despite the inherently structured nature of code, 77.2% of studies treat code as plain text. This preference is driven by the scalability and performance of Transformer-based LLMs. However, research indicates that graph-based modeling—using Abstract Syntax Trees (ASTs) or Data Flow Graphs—is superior for preserving semantic integrity during complex migrations.

The Rise of Prompt Engineering

There is a clear historical pivot from training specialized models from scratch (prevalent in 2020) to fine-tuning and Prompt Engineering (dominant by 2024). Researchers are increasingly "guiding" general-purpose LLMs rather than building task-specific architectures.

The Execution Gap

Static metrics like BLEU, which measure text overlap, are increasingly viewed as insufficient. A high BLEU score does not guarantee that code will compile or run correctly. Consequently, Computational Accuracy (CA)—the ability of translated code to pass unit tests—has become the mandatory standard for quality.

  • Bidirectional Dominance: 71.9% of studies focus on bidirectional translation (e.g., Java ↔ Python), highlighting the need for cross-platform interoperability.
  • Granularity: 73.7% of research focuses on the function or method level. Translating entire repositories remains a significant challenge.
  • Language Trends: While Java, Python, and C++ remain the "Big Three," there is growing interest in Rust and Go, reflecting industry shifts toward memory safety and cloud-native architectures.
  • Post-processing: Only 40% of studies utilize post-processing (e.g., iterative feedback loops or compiler diagnostics), yet these methods significantly improve final code executability.

Future Directions

The review identifies four critical gaps for future research:

  1. Repository-Level Translation: Moving beyond isolated functions to incorporate project-wide dependencies and context.
  2. Low-Resource Languages: Developing techniques for languages like OCaml or Julia that lack massive parallel corpora.
  3. Security: Mitigating the risk of translation models injecting insecure code patterns or logic errors.
  4. Agentic Workflows: Transitioning toward Multi-Agent Systems where specialized LLM agents collaborate as translators, testers, and debuggers to ensure functional equivalence.

Summary updated

Abstract

Code translation aims to convert code from one programming language to another automatically. It is motivated by the need for multi-language software development and legacy system migration. In recent years, neural code translation has gained significant attention, driven by rapid advancements in deep learning and large language models. Researchers have proposed various techniques to improve neural code translation quality. However, to the best of our knowledge, no comprehensive systematic literature review has been conducted to summarize the key techniques and challenges in this field. To fill this research gap, we collected 57 primary studies covering the period 2020~2025 on neural code translation. These studies are analyzed from seven key perspectives: task characteristics, data preprocessing, code modeling, model construction, post-processing, evaluation subjects, and evaluation metrics. Our analysis reveals current research trends, identifies unresolved challenges, and shows potential directions for future work. These findings can provide valuable insights for both researchers and practitioners in the field of neural code translation.
BibTeX
@misc{chen2025systematic,
  author = {Xiang Chen and Jiacheng Xue and Xiaofei Xie and Caokai Liang and Xiaolin Ju},
  title = {{A Systematic Literature Review on Neural Code Translation}},
  year = {2025},
  eprint = {2505.07425},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2505.07425},
}
Figure