Software engineering is increasingly being reframed as a linguistic challenge. Traditionally, tools for summarizing code, generating API sequences, or reviewing pull requests relied on rigid, rule-based systems or keyword retrieval. However, a growing body of research treats source code as a structured language that can be "translated" into natural language—and vice versa—using neural sequence-to-sequence (seq2seq) architectures.
The Translation Metaphor
The core intuition behind this research direction is that programming languages, despite their formal syntax, share the sequential and semantic properties of natural languages. As Song et al. note in their survey, code summarization can be viewed as a machine translation task where source code is encoded into a latent vector and decoded into a human-readable description.
This approach was pioneered by Iyer et al., who introduced CODE-NN. By using an LSTM with an attention mechanism, the authors demonstrated that a model could learn to "read" C# and SQL snippets and generate functional summaries without manual rule-crafting. This framing extends beyond simple summaries: Loyola et al. applied the same logic to code "diffs," treating the differences between versions of a program as a source language to be translated into commit messages.
Bridging Intent and Implementation
If code can be translated into text, the inverse is also true: natural language intent can be translated into executable code. This is particularly useful for API learning, where developers know what they want to do (e.g., "parse XML") but not the specific sequence of library calls required.
Gu et al. framed this as a translation problem, mapping natural language queries to ordered sequences of API calls. This captures the deep semantics and specific ordering of programming tasks that keyword search often misses. More recently, Hadi et al. showed that fine-tuning large pre-trained models like CodeBERT on millions of code snippets significantly boosts the accuracy of these translations, bridging the gap between developer intent and functional execution.
Recovering Lost Abstractions
The seq2seq paradigm also excels at recovering information lost during the software lifecycle. For example, when code is compiled, human-readable variable names and types are stripped away. Chen et al. proposed DIRTY, a Transformer-based model that treats decompilation as a translation task. By viewing tokenized decompiled code as a sequence, the model can predict original variable names and complex types, capturing long-range dependencies without the need for complex abstract syntax trees.
Similarly, Watson et al. developed ATLAS, which treats the generation of unit test assertions as a translation task. By encoding the context of a test and the method being tested, the model learns to "translate" that context into a formal assertion statement, effectively predicting the developer's intended test oracle.
From RNNs to Foundation Models
The evolution of these models mirrors broader trends in natural language processing. Early work, such as the question generation model by Reddy et al., relied on Recurrent Neural Networks (RNNs) and LSTMs to capture sequential dependencies. Gao et al. further refined these seq2seq models with "copy" and "coverage" mechanisms to handle technical identifiers in Stack Overflow snippets.
The field has since shifted toward Transformer-based architectures. Norouzi et al. demonstrated that standard Transformers, when combined with target autoencoding on monolingual code data, can achieve state-of-the-art results in code generation with minimal task-specific bias. Furthermore, Blloshmi et al. showed that pre-trained denoising autoencoders like BART can be adapted for complex labeling tasks by framing them as generation problems.
This shift suggests that these models are doing more than just surface-level pattern matching. Jin and Rinard argue that language models trained on program synthesis tasks develop emergent internal representations of formal program semantics, effectively simulating execution to satisfy the constraints of the input specification.
Key Foundational Work
Generating Natural Language Question-Answer Pairs from a Knowledge Graph Using a RNN Based Question Generation Model (2017) — contributed 2 concept elements to this direction.
Summarizing Source Code using a Neural Attention Model (2016) — contributed 1 concept elements to this direction.
Deep API Learning (2016) — contributed 1 concept elements to this direction.