Abstract
As an integral part of source code files, code comments help improve program readability and comprehension. However, developers sometimes do not comment on their program code adequately due to the incurred extra efforts, lack of relevant knowledge, or unawareness of the importance of code commenting. As a result, code comments can be inadequate, absent, or even mismatched with source code, which affects the understanding, reuse, and maintenance of software. To solve these problems, researchers have focused on generating code comments automatically. This work conducts a survey of automatic code commenting research, analyzing challenges, research frameworks, and the classification of representative algorithms, including their design principles, strengths, and weaknesses. We also provide an overview of quality assessment and summarize future directions for the field.
Introduction
Software maintenance and reuse are heavily dependent on high-quality documentation. Yet, developers often skip commenting because it is time-consuming or they lack context during active development. The challenge of Automatic Code Commenting is not merely describing what a line of code does, but explaining the developer's intent. Bridging the semantic gap between highly structured programming languages and fluid natural language requires an understanding of both program flow and logic.
Taxonomy of Generation Algorithms
Research into automatic comment generation is categorized into three primary algorithmic families: Information Retrieval (IR), Deep Learning (DL), and Heuristic/Structural models.
- Information Retrieval (IR) Based: These methods act as search engines, using techniques like the Vector Space Model (VSM) or Latent Semantic Indexing (LSI) to find similar code snippets in large repositories (like GitHub) and "borrowing" their existing comments.
- Deep Learning (DL) Based: This represents the current state-of-the-art, treating commenting as a Machine Translation problem (e.g., translating Python to English). Key architectures include Encoder-Decoder (Seq2Seq) models, Attention Mechanisms to focus on specific code elements, and Tree-based models (RvNN/CNN) that process Abstract Syntax Trees (ASTs) to capture logic hierarchies.
- Heuristic & Structural Models: These use specialized models like the Software Word Usage Model (SWUM) to extract "action-theme" relationships from code identifiers to build template-based descriptions.
Data Sources and Evaluation
There is currently no single "ImageNet" for code comments, making cross-paper comparisons difficult. Researchers typically draw from open-source repositories (GitHub), Q&A sites (Stack Overflow), and developer communications (bug reports).
Evaluation remains a significant hurdle. While translation metrics like BLEU and METEOR are standard, they often fail to capture the true semantic utility of a comment. A comment might have a low BLEU score but still be perfectly accurate and helpful to a human. Consequently, human evaluation for informativeness and interpretability remains the gold standard.
Evolution of the Field
The landscape of code comment generation has shifted dramatically over the last decade. Early research was predominantly IR-based or relied on heuristic templates. However, as shown in the publication trends, there has been an explosion in Deep Neural Network (DNN) approaches since 2015.
Hybrid models—those that combine structural analysis (like ASTs) with sequence-to-sequence learning—consistently outperform models that treat code as plain text. While DL-based methods are better at generalizing, they can occasionally produce "hallucinations"—grammatically correct English that is factually incorrect regarding the code's logic.
Future Directions
To advance the field, researchers must address dataset fragmentation by establishing unified, high-quality benchmarks. Furthermore, current models often suffer from "context blindness," looking only at single functions. Future work needs to incorporate "cross-file" context to understand how a specific function relates to the broader system. The next major leap is expected to come from better "fusion" models that combine the reliability of IR-based retrieval with the creative generation capabilities of Deep Learning.
Summary updated