Effective Bug Triage Based on Historical Bug-Fix Information

Hao Hu, Hongyu Zhang, Jifeng Xuan, Weigang Sun

TL;DR

This paper tackles the challenge of efficiently assigning bug reports to the right developers in large software projects. They introduce BugFixer, a novel approach that leverages historical bug-fix data to build a Developer-Component-Bug network, outperforming traditional machine learning methods, especially as team sizes grow.

What Problem Does This Paper Address?

In large software projects, teams receive a deluge of bug reports. Manually assigning these reports to developers with the right expertise is a time-consuming and error-prone process. This "bug triage" phase is critical, as misassignments can significantly delay bug resolution. Existing automated methods, often based on text classification and machine learning (like SVM and Naïve Bayes), work reasonably well for smaller teams but their effectiveness diminishes as the number of developers increases. This paper aims to bridge this gap by proposing a more scalable and effective bug triage method for large-scale projects.

Methods

The core of the proposed solution, BugFixer, lies in its novel approach to bug report assignment. Instead of directly classifying bugs to developers, BugFixer constructs a Developer-Component-Bug (DCB) network. This network models relationships between:

  • Developers and the Components (e.g., Java packages) they have worked on.
  • Components and the Bugs that were fixed within them.

This DCB network captures valuable "who fixed what, where" knowledge. When a new bug report arrives, BugFixer first computes its similarity to existing bug reports using a modified Vector Space Model (VSM). This VSM incorporates not only the bug report's text (summary and description) but also information from the source file names associated with previously fixed bugs, offering richer context.

The similarity scores are then used to traverse the DCB network, calculating relevance scores between the new bug report and potential developers. BugFixer outputs a ranked list of developers, with those having higher relevance scores being more likely candidates to fix the bug.

A key innovation in BugFixer is its novel tokenization algorithm for preprocessing bug report text. This algorithm goes beyond standard methods by handling abbreviations and composed words, and it also leverages WordNet and Java naming conventions to extract more meaningful terms.

Datasets

The authors evaluated BugFixer on:

  • Three large-scale open-source projects: Eclipse, Mozilla, and Netbeans.
  • Two industrial projects: EBCP and DMGR (company name undisclosed).

For these projects, they collected bug reports and associated bug-fixing commits from version control systems. The components in the DCB network were defined at the package level for Eclipse and Netbeans, and by grouping files based on their paths for Mozilla.

Key Insights and Ideas

  • Team size matters: The effectiveness of traditional bug triage methods (like SVM and Naïve Bayes) significantly degrades as the number of developers in a project increases. This highlights the need for methods that scale better.
  • Context is king: Incorporating source file names into bug similarity calculations provides valuable supplementary information that improves the accuracy of identifying related bugs.
  • Networked expertise: The DCB network effectively models developer expertise and bug characteristics by linking developers to components and components to bugs. This structured approach allows for more nuanced recommendations than simple text-based similarity.
  • Beyond text: While text analysis is crucial, BugFixer's success suggests that understanding the structural relationships within a software project (components) and historical developer activity is equally, if not more, important for effective bug triage.

Key Results

BugFixer demonstrated strong performance, particularly on large projects:

  • Eclipse: BugFixer achieved a Recall@1 of 42.36% and Recall@5 of 73.85%. This means the actual developer was ranked among the top 1 in over 42% of cases and among the top 5 in over 73% of cases.
  • Comparison: BugFixer consistently outperformed SVM and Naïve Bayes based methods on large projects, showing improvements of 6-182% in Recall@k. For smaller projects, it achieved comparable performance to SVM and better than Naïve Bayes.
  • Component of BugFixer: Both the proposed bug similarity method and the DCB network recommendation technique showed significant improvements in performance when integrated into BugFixer.

Limitations and Future Work

The authors acknowledge several limitations:

  • New developers: BugFixer's performance can be affected when bugs are fixed by developers who have no prior history in the dataset (new developers).
  • Brief bug reports: Very short or poorly written bug reports can hinder similarity comparisons, as they may lack sufficient detail or use ambiguous language.
  • Semantic similarity: The current method doesn't account for semantic similarity between words (e.g., "crash" vs. "exception"), which could lead to missed connections.
  • Data quality: Links between bugs and components in version control systems can sometimes be noisy or missing.

Future work will focus on addressing these limitations by exploring more features, improving semantic understanding, and investigating the impact of data quality on bug triage. They also plan to conduct larger-scale evaluations across a wider variety of projects.

Summary updated

Abstract

For complex and popular software, project teams could receive a large number of bug reports. It is often tedious and costly to manually assign these bug reports to developers who have the expertise to fix the bugs. Many bug triage techniques have been proposed to automate this process. In this paper, we describe our study on applying conventional bug triage techniques to projects of different sizes. We find that the effectiveness of a bug triage technique largely depends on the size of a project team (measured in terms of the number of developers). The conventional bug triage methods become less effective when the number of developers increases. To further improve the effectiveness of bug triage for large projects, we propose a novel recommendation method called Bug Fixer, which recommends developers for a new bug report based on historical bug-fix information. Bug Fixer constructs a Developer-Component-Bug (DCB) network, which models the relationship between developers and source code components, as well as the relationship between the components and their associated bugs. A DCB network captures the knowledge of "who fixed what, where". For a new bug report, Bug Fixer uses a DCB network to recommend to triager a list of suitable developers who could fix this bug. We evaluate Bug Fixer on three large-scale open source projects and two smaller industrial projects. The experimental results show that the proposed method outperforms the existing methods for large projects and achieves comparable performance for small projects.
BibTeX
@inproceedings{hu2014effective,
  author = {Hao Hu and Hongyu Zhang and Jifeng Xuan and Weigang Sun},
  title = {{Effective Bug Triage Based on Historical Bug-Fix Information}},
  booktitle = {ISSRE},
  year = {2014},
  doi = {10.1109/issre.2014.17},
}
Figure