Automatic Code Review by Learning the Structure Information of Code Graph

Ying Yin, Yuhai Zhao, Yiming Sun, Chen Chen

TL;DR

This paper tackles the challenge of inefficient manual code reviews by proposing an automated code review system that leverages both code sequence and structural information. Their novel PDG2Seq algorithm serializes program dependency graphs into sequences, which are then fed into a CodeBERT-based model (crBERT) to improve code modification suggestions and overall review efficiency.

What Problem Does This Paper Address?

The ever-increasing volume and complexity of software code make manual code review a time-consuming and labor-intensive process. While existing automated code review tools often rely solely on code sequence information, they overlook the rich structural and semantic information embedded within a program's logic. This paper aims to bridge this gap by developing an automated code review model that effectively integrates program structure (via Program Dependency Graphs) with code sequence data to enhance the accuracy and efficiency of code review tasks.

Methods

The core of this work lies in two main contributions:

  • PDG2Seq Algorithm: This novel algorithm serializes Program Dependency Graphs (PDGs) into a unique, lossless graph code sequence. This sequence retains both the structural and semantic information of the PDG, overcoming the limitations of traditional graph representation methods for deep learning models. The algorithm is inspired by minimal DFS encoding and incorporates specific rules for handling forward and backward edges to ensure a consistent and informative representation.
  • crBERT Model: Built upon the pre-trained CodeBERT architecture, this model is designed for automated code review. It consists of:
  • CodeEncoder: Extracts semantic features from code sequences.
  • PDGEncoder: Extracts structural and semantic features from the graph sequences generated by PDG2Seq.
  • ReviewEncoder (for Task 2): Extracts features from reviewer comments.
  • Decoder: Generates the modified code.

The crBERT model fuses these features to learn a mapping from input code and (optionally) review comments to the corrected code, aiming to produce suggestions that are closer to what a human reviewer would provide.

Datasets

The study utilizes an open-source dataset previously introduced by Tufano et al. This dataset comprises triplets of <original code, reviewer comment, modified code> crawled from code review platforms like Gerrit and GitHub. The authors performed code abstraction on this dataset and filtered out samples that couldn't be converted into PDGs. The final dataset consists of 12,727 training, 1,584 validation, and 1,598 testing samples.

Key Insights and Ideas

  • Structure Matters: The paper strongly advocates for the importance of code's structural information (control flow, data dependencies) in addition to its sequential representation for effective automated code review.
  • PDG Serialization is Key: The PDG2Seq algorithm provides a crucial bridge, converting complex graph structures into a format that deep learning models can effectively process while preserving essential information.
  • CodeBERT's Power: Leveraging a pre-trained model like CodeBERT, which already understands code and natural language, provides a strong foundation for building sophisticated code review tools.
  • Task Specialization: The paper addresses two distinct tasks: (1) automatic code modification based on code changes, and (2) automatic code modification incorporating reviewer comments, demonstrating the model's versatility.

Key Results

The proposed crBERT model, integrating PDG2Seq, demonstrated significant improvements over existing state-of-the-art methods across various metrics:

  • Task 1 (Code Modification): Achieved improvements in BLEU scores, with gains of up to 8.8% over the 1-encoder model.
  • Task 2 (Code Modification with Comments): Showed improvements of up to 3.6% in BLEU scores over the 2-encoder model.
  • General Performance: Consistently outperformed baseline models (Seq2Seq, Transformer) in terms of BLEU, Levenshtein distance, and ROUGE-L metrics. For instance, crBERT showed a 7.7% and 2.6% point improvement on Task 1 and Task 2, respectively, compared to the CodeBERT baseline without graph features.

Limitations and Future Work

  • Code Language Specificity: The current work focuses on Java. Future work could extend the approach to support other programming languages like Python and C++.
  • Granularity: The review granularity is currently at the method level. The authors suggest exploring finer-grained (statement-level) or coarser-grained reviews.
  • Broader Applications: The proposed code representation and learning techniques could be applied to other software engineering tasks such as defect detection, automatic code generation, and annotation.
  • Dataset Size: While the dataset is substantial, the authors filtered out samples that couldn't be converted to PDGs, which might introduce a bias. Further research could explore methods to handle such cases.

Summary updated

BibTeX
@inproceedings{yin2023automatic,
  author = {Ying Yin and Yuhai Zhao and Yiming Sun and Chen Chen},
  title = {{Automatic Code Review by Learning the Structure Information of Code Graph}},
  booktitle = {Sensors},
  year = {2023},
  url = {https://openreview.net/forum?id=MllOAEkxKX},
}
Figure