DeBGUer: A Tool for Bug Prediction and Diagnosis

Amir Elmishali, Roni Stern, Meir KalechAAAI 2019 · DOI

TL;DR

DeBGUer is an open-source, web-based tool that streamlines the debugging process by combining Machine Learning-based bug prediction with automated fault diagnosis. By implementing the "Learn" and "Diagnose" phases of the LDP (Learn, Diagnose, and Plan) paradigm, it helps developers identify likely-buggy components before they fail and pinpoint the root cause of failed tests with high precision.

Introduction

Software debugging remains one of the most time-consuming and expensive stages of the development lifecycle. Traditionally, developers rely on manual inspection or basic testing suites to find and fix bugs. While tools for "Fault Localization" (finding where a bug is after a test fails) exist, they often lack the historical context of a project's development.

The authors identify a critical gap: most tools either predict where bugs might occur (Fault Prediction) or diagnose why a test failed (Fault Diagnosis), but rarely both. DeBGUer bridges this gap by using historical data from version control and issue trackers to guide the diagnostic process, making it easier for engineers to isolate the "root cause" of a failure.

Methodology

DeBGUer is built on the LDP paradigm, which consists of three AI-driven components:

The LDP (Learn, Diagnose, and Plan) paradigm workflow implemented in DeBGUer.
  1. Fault Predictor (Learn): This is a classification model that analyzes 405 different features of the code—including complexity metrics (McCabe/Halstead), object-oriented measures, and "process" features (like the age of a file or the number of developers who touched it). It uses a Random Forest algorithm to estimate the probability that any given component (class or method) contains a bug.
  2. Diagnoser (Diagnose): When tests fail, DeBGUer uses an extension of the Barinel algorithm. This Bayesian approach combines the "traces" of failed tests (which parts of the code were executed) with the probabilities from the Fault Predictor.
  3. Automated Labeling: To train the model without manual effort, the tool automatically links Jira issue trackers with Git commit histories. It identifies "bug-fix" commits and labels the modified files as the historical root causes.

The system is implemented as a client-server application using PHP, AngularJS, and the WEKA machine learning library.

Datasets and Evaluation

The researchers evaluated DeBGUer using 12 large-scale open-source Java projects from the Apache ecosystem, including Ant, Flink, IO, JClouds, and MyFaces. The evaluation involved thousands of commits and issues, providing a robust real-world benchmark for the tool's predictive and diagnostic capabilities.

Key Insights

  • Priors Matter: The most significant innovation is using the Fault Predictor's output as a "prior" for the Bayesian diagnosis. Instead of assuming every component is equally likely to be broken, the diagnoser "leans" toward components that the ML model already flagged as suspicious based on historical patterns.
  • Heuristic Labeling: Since manually labeling every bug in history is impossible, the authors used "all modified" and "most modified" heuristics to identify faulty components in past commits. They found that even these simple heuristics provide enough signal for high-accuracy prediction.
  • Web-Based Accessibility: Unlike many academic tools that are command-line only, DeBGUer provides a visual interface that allows developers to browse their project's package structure and see "heat maps" of bug probability.
The DeBGUer interface displaying fault prediction results for various code components.

Key Results

  • High Prediction Accuracy: For most projects, the Fault Predictor achieved precision scores above 0.9 and recall scores generally above 0.68. This means when the tool flags a component as buggy, it is almost always right.
  • Efficient Diagnosis: The "Top k" metric (how many suggestions a developer must check to find the real bug) was impressively low. In the IO project, the Top k was 1.0, meaning the very first suggestion was the actual bug. Across all projects, the Top k ranged from 1 to 4.24, significantly reducing the search space for developers.
  • Robustness: The tool performed well even when failures were "intermittent" (simulated by adding a probability that a faulty component might still pass a test).

Limitations and Future Work

  • The Missing "Plan": The current version of DeBGUer only implements the "Learn" and "Diagnose" parts of LDP. The "Plan" phase—which would involve automatically generating new tests to further narrow down a diagnosis—is not yet implemented.
  • User Study Needed: While the quantitative results are strong, the authors acknowledge the need for a formal user study to see how much time professional developers actually save when using the tool in their daily workflow.
  • Language Support: The current implementation is tailored for Java/Maven environments; expanding to other ecosystems (like Python or C++) remains a future goal.

Summary updated

BibTeX
@inproceedings{elmishali2019debguer,
  author = {Amir Elmishali and Roni Stern and Meir Kalech},
  title = {{DeBGUer: A Tool for Bug Prediction and Diagnosis}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2019},
  doi = {10.1609/aaai.v33i01.33019446},
}
Figure