Encoding Version History Context for Better Code Representation

Huy Nguyen, Christoph Treude, Patanamon Thongtanunam

Introduction

In modern software engineering, code is rarely a static artifact; it is a living entity that evolves through iterative changes, refactorings, and bug fixes. However, most state-of-the-art deep learning models for code representation—such as CodeBERT and ASTNN—treat source code as a single, isolated snapshot. By ignoring the historical trajectory of a program, these models miss critical semantic context that human developers rely on, such as commit messages, previous implementations, and the evolution of logic over time.

This paper explores the hypothesis that "teaching" models about a program’s version history can significantly enhance their ability to understand code semantics. By incorporating the sequence of changes a code snippet has undergone, the researchers demonstrate substantial performance improvements in downstream tasks like code clone detection and software classification.

The Problem: The "Short-Sightedness" of Static Models

When a developer encounters a complex function, they often use tools like git blame or git log to understand why certain decisions were made. A current version of a function might look entirely different from its original form due to refactoring, yet its core purpose remains the same.

Existing models typically process only the latest version of a snippet. This lack of historical context creates a gap in understanding: two functions might be semantically identical (clones) but appear syntactically distinct in their current state. Without access to their shared ancestry, a model may fail to recognize their relationship. The challenge lies in systematically encoding this temporal evolution into modern neural architectures.

Methodology

The study evaluates the impact of version history using two distinct model architectures:

  1. ASTNN: A model designed to capture structural information by processing Abstract Syntax Trees (ASTs).
  2. CodeBERT: A transformer-based model pre-trained on large-scale natural language and programming language corpora.

Encoding Evolution

To integrate history, the researchers extracted multiple historical versions of specific methods. They experimented with three primary aggregation techniques to merge past representations with the current code vector:

  • Concatenation: Joining the vector representations of the current and historical versions into a single feature set.
  • Max-pooling: Identifying and selecting the most prominent features across all versions to create a representative summary.
  • Concatenation of Absolute Difference: Calculating the mathematical delta between the current and past versions to explicitly highlight what changed during the code's evolution.

Beyond version history, the researchers also incorporated "extra-curricular" context, including the call hierarchy (the network of functions that invoke the snippet) and the temporal age (the number of days the code has existed in the repository).

Experimental Setup

The researchers utilized the SeSaMe dataset, which consists of semantically similar Java methods sourced from 11 prominent open-source projects, including openjdk11, guava, and caffeine. Using tools like PyDriller and Lizard, they extracted 10,531 code versions for 1,679 unique Java methods. The dataset reflects real-world complexity, with some methods maintaining a long history of over 26 versions.

Key Results and Insights

The inclusion of version history yielded significant improvements across two primary tasks:

1. Code Clone Detection

Version history acts as a powerful signal for similarity. Adding historical context improved the F1 score of CodeBERT by 15% (rising from 0.667 to 0.769) and ASTNN by 7%. The most dramatic gains occurred when history was combined with metadata: when the "number of days" a snippet existed was added to the version history, CodeBERT’s F1 score for clone detection increased by 27% over the baseline.

2. Code Classification

For the task of identifying which project a specific code snippet belongs to, the combination of version history and call hierarchy allowed ASTNN to achieve a 27% increase in accuracy. This suggests that a function's "social circle" (its callers) and its "life story" (its history) are highly indicative of its project-specific identity.

3. Model-Specific Synergy

The research found that the optimal aggregation strategy is model-dependent. While ASTNN performed best with simple concatenation, CodeBERT often benefited more from the "absolute difference" method, which focuses the model's attention on the nature of the changes rather than just the raw state of the code.

Limitations and Future Work

While the results are promising, several challenges remain:

  • Token Limits: Transformer models like CodeBERT have strict input length limits (e.g., 512 tokens). Long version histories risk being truncated, potentially losing the most valuable early context.
  • Computational Overhead: Processing multiple versions of a file increases the computational cost linearly. Future research should investigate selective encoding—identifying which specific historical versions are most informative to avoid redundant processing.
  • Scalability: This preliminary study focused on a relatively small dataset of 1,679 methods. Larger-scale evaluations are necessary to confirm these findings across more diverse programming languages and project types.

Conclusion

By shifting the focus from static snapshots to evolutionary history, this research demonstrates that the "story" of code is just as important as its current state. Integrating version history and structural context provides a more holistic representation of software, paving the way for more intelligent and context-aware AI tools for software engineering.

Summary updated

BibTeX
@inproceedings{nguyen2024encoding,
  author = {Huy Nguyen and Christoph Treude and Patanamon Thongtanunam},
  title = {{Encoding Version History Context for Better Code Representation}},
  booktitle = {MSR},
  year = {2024},
  doi = {10.1145/3643991.3644929},
}
Figure