Abstract
Source code summarization is a critical task for software maintenance, requiring models to understand both the sequential and structural nature of code. We propose a novel hybrid architecture that combines Graph Convolutional Networks (GCNs) with Transformers. By modifying Abstract Syntax Trees (ASTs) to include sibling relationships and using GCNs to enrich node representations, our model captures deep structural logic. A subsequent Transformer encoder-decoder processes the linearized, structurally-aware sequence to generate natural language summaries. Evaluations on Java and Python benchmarks show our model achieves state-of-the-art results and produces more fluent, relevant summaries according to human experts.
Introduction
Source code is unique because it is simultaneously sequential (read line-by-line) and structural (organized into nested blocks, loops, and dependencies). Existing summarization models usually prioritize one aspect over the other. Text-based approaches using Transformers or RNNs often miss the deep logic of code structure, while tree-based approaches using Graph Neural Networks can struggle with long-range sequential flow. This research bridges the gap by developing a hybrid model that leverages the strengths of both structural and sequential processing.
Methodology
The proposed pipeline captures both structural and sequential information through a multi-stage process:
- Modified AST (mAST): Standard Abstract Syntax Trees (ASTs) represent parent-child relationships. The authors enhance this by adding "sibling edges" between adjacent statements at the same level, allowing the model to perceive the flow between consecutive blocks.
- Graph Convolutional Network (GCN): GCN layers are applied to the mAST to "smooth" node representations. This ensures that a node's embedding incorporates information from its parents, children, and siblings.
- Linearization: The GCN-enriched tree is converted into a sequence via pre-order traversal.
- Transformer Encoder-Decoder: This sequence of structurally-aware nodes is fed into a standard Transformer. The prior GCN processing makes it easier for the Transformer's attention mechanism to group related code blocks.
Key Insights
- Structural Similarity and Attention: By using a GCN before the Transformer, nodes that are logically close (e.g., a variable and its controlling loop) develop similar vector representations. This significantly improves the effectiveness of the Transformer's self-attention mechanism.
- The Importance of Siblings: Standard ASTs treat consecutive lines as distant relatives. Adding sibling edges explicitly teaches the model that the order of statements within a block is a critical structural feature.
- Optimal Architecture: Ablation studies revealed that placing GCN layers at the beginning of the encoder is more effective than at the end, acting as a structural feature extractor. However, using more than two GCN layers leads to "over-smoothing," where nodes lose their unique identity.
Experimental Results
The model, mAST+GCN, was evaluated on large-scale Java (69,708 samples) and Python (55,538 samples) datasets. It outperformed several high-profile baselines, including DeepCom and TransRel.
- Quantitative Performance: The model achieved a BLEU score of 45.49 on the Java dataset and 32.82 on Python, setting a new state-of-the-art.
- Human Evaluation: Java experts rated the generated summaries higher in Fluency, Relevance, and Coverage compared to baseline models.
- Attention Analysis: Visualizing attention weights confirmed that the mAST+GCN model successfully focuses on distinct hierarchical blocks, appearing as clear rectangles in heatmaps, whereas baselines produced noisier patterns.
Limitations and Future Work
While effective, the model's performance is tied to the quality of the AST parser; syntactically incorrect code or languages with limited parsing support may pose challenges. Additionally, the inclusion of GCN layers and sibling edges increases preprocessing and training complexity. Future research could explore the model's generalizability to lower-level languages like C or functional languages like Haskell, where AST structures differ significantly.
Summary updated