A Unified Active Learning Framework for Annotating Graph Data with Application to Software Source Code Performance Prediction

Peter Samoaa, Linus Aronsson, Antonio Longa, Philipp Leitner, Morteza Haghir Chehreghani

Abstract

This paper presents a unified active learning framework designed to predict software performance by representing source code as Flow-Augmented Abstract Syntax Trees (FA-AST). By leveraging graph representation learning and uncertainty-based query strategies, the framework significantly reduces the labeling effort required for performance engineering, achieving high predictive accuracy with minimal benchmarked data.

Introduction

Predicting software performance metrics, such as execution time, is a critical task in modern software engineering. Early detection of performance bottlenecks can save significant development time and operational costs. However, training machine learning models for this purpose typically requires vast amounts of labeled data, which in this context means executing thousands of programs under various workloads to measure their speed. This benchmarking process is prohibitively expensive in terms of time, electricity, and computational resources.

To address this "data bottleneck," this research introduces a unified active learning (AL) framework. Instead of labeling the entire dataset, the system intelligently identifies and requests labels only for the most "informative" code snippets. This approach allows models to achieve high predictive accuracy while using only a small fraction of the total available data.

Methodology

Graph Representation: FA-AST

The core of the framework lies in how source code is represented. Traditional models often struggle with the non-linear logic of software. The authors propose using Flow-Augmented Abstract Syntax Trees (FA-AST). The process begins by parsing Java source code into a standard Abstract Syntax Tree (AST). To capture the program's actual logic, this tree is augmented with additional edges representing data flow (variable usage) and control flow (branching and loops).

The pipeline from raw Java code to a Flow-Augmented AST graph.

Graph Embedding and Active Learning

Because regression models cannot process graphs directly, embedding techniques like Graph2Vec or Graph Neural Networks (GNNs) are used to compress these complex structures into fixed-length vectors.

The framework then enters an active learning loop using a Gaussian Process Regressor (GPR). GPRs are particularly useful because they provide a measure of uncertainty for their predictions. This uncertainty is used by various query strategies to select the next samples for benchmarking:

  • Variance: Selecting samples where the model's uncertainty is highest.
  • Coreset: Selecting samples that are most representative of the overall unlabeled pool.
  • Query-by-Committee (QBC): Utilizing multiple models and selecting samples where disagreement is maximized.
The unified active learning framework loop.

Experimental Evaluation

Datasets

The framework was evaluated on two real-world datasets:

  1. OSSBuilds: Performance data from four open-source projects (systemDS, H2, Dubbo, and RDF4J) collected from continuous integration systems.
  2. HadoopTests: A complex dataset created by the authors through controlled execution of unit tests from the Apache Hadoop project.

The study also compared File-level parsing against System-level parsing. While system-level parsing creates larger graphs by resolving references across the entire project, it provides a richer context that generally leads to better embeddings.

Key Insights and Results

Performance and Efficiency

The best-performing models achieved a Pearson correlation score of approximately 0.75 on the HadoopTests dataset. Crucially, the active learning strategies reached this level of performance much faster than random sampling, proving that intelligent data selection significantly reduces the cost of performance engineering.

Representation Learning

A surprising finding was that Graph2Vec, an unsupervised method, often outperformed supervised GNNs. The authors attribute this to the "tree-like" nature of source code graphs; Graph2Vec appears better at capturing deep, branching paths compared to the local message-passing mechanisms of standard GNNs. Additionally, a "transductive" approach—where the embedding process sees the features of the test data without their labels—significantly improved results by helping the model map the latent space more effectively.

Conclusion and Future Work

The proposed framework demonstrates that active learning combined with flow-augmented graph representations is a powerful tool for software performance prediction. While highly effective, current limitations include incomplete support for certain Java constructs (e.g., switch statements) and the high computational cost of processing large system-level graphs. Future research will focus on more efficient graph pruning techniques and exploring alternative regression models that may scale better to massive datasets.

Summary updated

Abstract

Replication Package for <em>A Unified Active Learning Framework for Annotating Graph Data with Application to Software Source Code Performance Prediction</em>
BibTeX
@inproceedings{samoaa2023unified,
  author = {Peter Samoaa and Linus Aronsson and Antonio Longa and Philipp Leitner and Morteza Haghir Chehreghani},
  title = {{A Unified Active Learning Framework for Annotating Graph Data with Application to Software Source Code Performance Prediction}},
  booktitle = {Zenodo},
  year = {2023},
  doi = {10.5281/zenodo.7792485},
}

Loading related content...

Figure