TL;DR
This paper proposes "Software Intelligence" (SI) as the future direction for mining software engineering data, drawing inspiration from Business Intelligence. SI aims to provide software practitioners with timely, fact-based insights to support daily decision-making throughout a software system's lifecycle, moving beyond intuition and gut feelings.
What Problem Does This Paper Address?
Modern software development and maintenance often rely heavily on intuition and experience for critical decisions, such as when to release a system, which parts to refactor, or where to focus testing efforts. This haphazard approach leads to wasted resources and increased costs. While the field of Mining Software Repositories (MSR) has made strides in analyzing software data, its impact on practical decision-making is still limited. The authors argue that MSR needs to evolve towards providing actionable "Software Intelligence" to empower practitioners with data-driven insights, bridging the gap between research and practice.
Methods
The paper doesn't introduce new algorithms or methods but rather advocates for a shift in focus and application of existing MSR techniques. It emphasizes:
- Leveraging diverse data sources: Moving beyond traditional source code and bug repositories to include non-historical data like execution logs, developer interactions, and archived communications.
- Problem-driven approach: Shifting from applying off-the-shelf data mining techniques to first identifying software engineering problems and then adapting or developing suitable mining algorithms to address them. This includes potential collaboration with data mining experts.
- Integration into practice: Encouraging the integration of SI insights into practitioners' daily workflows, such as through IDE plugins and visualizations, to make them easily accessible and understandable.
Datasets
The paper discusses various types of software engineering data that can be mined, rather than specific datasets used for evaluation. These include:
- Historical Repositories: Source control repositories (e.g., Git, SVN), bug repositories (e.g., Bugzilla, Jira), and archived communications (e.g., mailing lists, emails).
- Run-time Repositories: Deployment logs.
- Code Repositories: Public code hosting platforms (e.g., Sourceforge, Google Code).
- Non-Historical Data: Execution logs, developer interaction data within IDEs, meeting notes, support call recordings, and online postings about software products.
Key Insights and Ideas
- Software Intelligence (SI) as the next evolution of MSR: SI is presented not just as analyzing data, but as providing actionable intelligence to support decision-making throughout the software lifecycle.
- Beyond the coding phase: A significant portion of MSR research has focused on the coding phase. SI needs to extend its support to other stakeholders like managers, testers, and support teams, covering the entire software lifecycle.
- The importance of non-historical data: While historical data is valuable, SI can be significantly enhanced by incorporating real-time and non-repository data sources.
- Bridging the research-practice gap: The paper stresses the need for SI to be intuitive, explainable, and integrated into practitioners' tools to ensure adoption.
- SI as a platform for research: SI can also serve as an enabling platform for future software engineering research, facilitating empirical studies and the development of evolutionary approaches.
Key Results
This paper is a position paper, so it does not present quantitative results from experiments. Instead, it outlines a vision and research agenda. The "key results" are conceptual:
- The proposal of "Software Intelligence" as a unifying concept for mining software engineering data.
- A roadmap for future research directions in MSR to enable SI, focusing on broader data sources, problem-driven techniques, and practical adoption.
- The identification of existing commercial products (e.g., Coverity, Pattern Insight) as early examples of SI in practice.
Limitations and Future Work
Limitations:
- The vision of SI is not yet a reality, and significant research and development are needed to achieve it.
- Privacy concerns are highlighted as a challenge when dealing with more personal and real-time data.
- The current reliance on heuristics for data collection can lead to noisy data.
Future Work:
- SI throughout the lifecycle: Expand MSR research beyond the coding phase to support all stakeholders and phases of a software project.
- Leveraging diverse data: Mine a wider range of software engineering data, including non-historical and real-time sources.
- Improving data collection: Develop better mechanisms for collecting high-quality data, potentially by influencing IDE and repository design.
- Developing effective mining techniques: Adopt a problem-driven methodology, collaborate with data mining experts, and develop new algorithms or adapt existing ones.
- Facilitating adoption: Make SI techniques intuitive, explainable, and integrate them into practitioners' tools (e.g., IDEs).
- Creating new roles: Establish roles for "curators" to maintain and ensure the quality of software repositories.
- Integrating with Business Intelligence: Explore selling software engineering decision-making as part of traditional BI platforms.
- Empirical evaluation: Research projects and papers should be evaluated based on their SI capabilities and demonstrated value to practitioners.
Summary updated