TL;DR
This paper introduces SeekChanges, a novel framework for change localization that goes beyond traditional methods by simultaneously identifying files to be modified and suggesting packages where new files should be created. By treating software as a hierarchical structure and employing structured information retrieval techniques, SeekChanges offers a more comprehensive solution for developers tackling diverse change requests.
What Problem Does This Paper Address?
Software development is an ongoing process, with maintenance and evolution requiring developers to handle numerous change requests. These requests can range from fixing bugs to implementing new features. A critical bottleneck in this process is change localization: pinpointing exactly where in the codebase changes need to be made. Existing automated tools often fall short because they are specialized for either bug fixing (identifying files to modify) or feature location (identifying existing code related to a feature), but struggle with the common scenario where a single request necessitates both modifying existing files and creating entirely new ones. This is particularly problematic for new feature requests, where developers need guidance on where to place new code, a task current methods largely ignore.
Methods
The core of the SeekChanges framework is its ability to handle both types of changes (modification and addition) within a unified approach. It models change localization as a structured information retrieval (IR) task, leveraging the hierarchical nature of Java projects (packages and files).
The process involves:
- Preprocessing: Formulating a query from the issue report (summary and description), cleaning it (stemming, stop-word removal, lowercasing), and tokenizing composite terms.
- Indexing: Extracting and indexing features from source code files.
- Classical IR Modeling: Using standard IR models like Vector Space Model (VSM) and Unigram Language Model (UM) to compute initial relevance scores between the query and source code files.
- Organizing Packages into Hierarchical Structures: Representing the Java project's package and file structure in a hierarchical, XML-like format.
- Structured IR Modeling: This is the novel part. Three structured IR strategies are investigated to compute scores for packages based on the scores of their contained files:
- ElementScoring: Scores elements based on their prior probability and the query's likelihood given the element's language model.
- Propagation: Scores parent elements by propagating scores from child elements, incorporating a propagation factor.
- Aggregation: Scores parent elements by aggregating the scores of their child elements.
These structured IR strategies aim to provide relevance scores for both individual files (for modification) and packages (for new file creation).
Datasets
The study utilized four open-source Java projects from the Apache Software Foundation: Myfaces Core, Pig, Wicket, and Zookeeper. These projects were chosen because they use Jira for issue tracking, which simplifies the extraction of truly modified and added files. The datasets were constructed by collecting all resolved change requests between two adjacent releases. A key finding from this dataset analysis was that newly created files constitute a significant portion (30-50%) of all changed files, underscoring the need for methods that can recommend new file locations.
Key Insights and Ideas
- Unified Approach is Crucial: The significant proportion of newly added files in change requests highlights that focusing solely on modifying existing code is insufficient. A system that can recommend both modification locations and new file placement is essential for practical change localization.
- Leveraging Hierarchy Matters: The hierarchical structure of software (packages containing files) is not just an organizational detail; it's a valuable signal for change localization. Structured IR techniques can effectively exploit this hierarchy.
- Structured IR Outperforms Classical IR: The proposed structured IR strategies (especially Propagation and Aggregation) consistently outperformed baseline classical IR models (VSM, UM) for both file and package recommendation.
- Propagation is a Promising Strategy: The Propagation method emerged as the most effective structured IR strategy, demonstrating strong performance across various metrics and datasets. Its adaptability with parameters also makes it a flexible choice.
- Computational Overhead is Manageable: The added computational cost of using structured IR strategies was found to be negligible, making them practical for real-world use.
Key Results
- The Propagation structured IR strategy was identified as the best-performed method, significantly outperforming baseline VSM and UM models for both file and package recommendation across all four projects.
- In terms of package recommendation (identifying where new files should be added), structured IR strategies showed good performance, with Propagation achieving particularly high scores (e.g., MAP scores up to 0.4278 for Myfaces and 0.4375 for Pig).
- When combined with a state-of-the-art bug localization tool (BugLocator), the Propagation strategy achieved competitive results for overall change localization, demonstrating its ability to integrate with existing powerful file-scoring mechanisms. For instance, using BugLocator with Propagation achieved a MAP of 0.3902 for Myfaces and 0.3806 for Pig in the "Propagation, add" scenario.
- The average execution time per query (ET/Qi) for the structured IR strategies was found to be low (e.g., 0.13s for Propagation on Myfaces), indicating efficient computation.
Limitations and Future Work
- External Validity Concerns: The study's datasets were limited to four Apache Java projects using Jira. The authors acknowledge that applying these methods to projects using different issue-tracking systems or to commercial projects might yield different results.
- Construct Validity Concerns: The representation of package content by simply concatenating source code files might not be optimal. Future work could explore richer representations, such as API-level descriptions.
- Limited Scope of Heuristics: The current approach primarily relies on textual similarity and historical bug data. Future work could incorporate additional heuristics like stack traces or contextual relationships to further improve accuracy.
- Granularity of Recommendations: The authors suggest expanding the scope to recommend changes at lower levels of abstraction, such as methods or variables, in addition to files and packages.
- Advanced IR Models: Exploring more advanced IR models (e.g., deep learning) for computing file relevance scores could potentially enhance the performance of the structured IR strategies.
Summary updated