Identification-Method Research for Open-Source Software Ecosystems

Zhifang Liao, Ningwei Wang, Shengzong Liu, Yan Zhang, Hui Liu, Qi ZhangSymmetry 2019 · DOI

TL;DR

This paper introduces a novel framework, TPKMF, to identify and analyze software ecosystems within the vast GitHub platform. By leveraging topic modeling and a new spectral clustering algorithm (CP-SC), the authors effectively map project relationships and uncover distinct software ecosystems, demonstrating that developer interaction plays a crucial role in their relevance and sustainability.

What Problem Does This Paper Address?

The rapid growth of open-source software (OSS) has led to the emergence of complex "software ecosystems" – collections of projects that coevolve due to shared developers and interdependencies. Identifying these ecosystems and understanding the relationships between projects within them is a significant challenge. Existing methods for analyzing project relevance often struggle with the sheer scale of platforms like GitHub, are computationally expensive, or rely on incomplete data. This paper aims to bridge this gap by proposing a scalable and effective method for identifying software ecosystems and analyzing project relevance within them.

Methods

The core of this research lies in two main contributions:

  • Topic-based Project Knowledge Metrics Framework (TPKMF): This framework extracts and fuses project information (description, language, etc.) and interactive information (commit messages, issue comments, etc.) from GitHub. It then uses Latent Dirichlet Allocation (LDA) to model topics within this text data, creating a topic-project relation matrix. This matrix is then used to construct a similarity matrix, which forms the basis of a project-relevance network.
  • Core Project-Spectral Clustering (CP-SC) Algorithm: To identify distinct ecosystems within the project-relevance network, the authors developed CP-SC. This algorithm improves upon standard spectral clustering by intelligently selecting initial cluster centers based on node conductance (a measure of how likely a random walk is to leave a cluster). This "core project" centric approach aims to create more robust and meaningful community divisions.

Datasets

The primary dataset used in this research is derived from GitHub. The authors collected data from:

  • GitHub project home pages: For project information (names, descriptions, file lists, attention data, etc.).
  • GHTorrent project and GitHub Archive project: For interactive information, specifically focusing on event data such as pushEvent, pullrequestEvent, issuesEvent, commitCommentEvent, issueCommentEvent, and pullrequestReviewCommentEvent.

The paper also uses publicly available datasets for evaluation and comparison:

  • Dolphin Social Network data
  • NCAA College Football Network data

Key Insights and Ideas

  • Developer Interaction is Key: The research strongly suggests that interactive information (e.g., commit messages, issue discussions) has a significantly larger impact on project relevance and ecosystem formation than static project information alone. This highlights the social and collaborative nature of OSS development.
  • Core Projects Anchor Ecosystems: The CP-SC algorithm's success in identifying ecosystems reinforces the idea that most software ecosystems are organized around one or a few "core projects" to which other projects are closely related or dependent.
  • Scalability Matters: The TPKMF and CP-SC algorithms are designed to handle the massive scale of platforms like GitHub, overcoming limitations of previous methods that struggled with large datasets.
  • Topic Modeling Bridges the Gap: LDA effectively translates unstructured text data from projects into a structured topic representation, enabling meaningful analysis of project relationships.

Key Results

  • The CP-SC algorithm successfully identified 35 distinct ecosystems within the analyzed GitHub projects, with about one-third of these aligning with findings from previous studies.
  • In performance comparisons against other community detection algorithms (LPA, CNM, FastUnfolding, Infomap, SC), CP-SC demonstrated a strong balance of accuracy (high NMI) and runtime, often outperforming others, especially on the GitHub dataset.
  • Pearson correlation analysis revealed a weak association (0.228) between project information and the project-relevance network, but a strong association (0.813) between interactive information and the project-relevance network.
  • The project-relevance network visualized using Gephi showed clear community structures, with prominent projects like nodejs/node, apple/swift, and mozilla-b2g/B2G playing central roles in their respective ecosystems.

Limitations and Future Work

  • New Project Data: The TPKMF's performance can be suboptimal when dealing with new projects that have limited historical information.
  • Evolution of Ecosystems: The current study focuses on identifying ecosystems at a specific point in time. The authors suggest exploring the evolution of these ecosystems as a future research direction.
  • Other Platforms: While the method is demonstrated on GitHub, its applicability to other open-source platforms could be further investigated.
  • Deep Learning: The authors plan to explore deep learning methods for analyzing project relevance and identifying influential projects.
  • Developer Recommendation: Future work may involve recommending suitable developers to new projects based on ecosystem analysis.

Summary updated

BibTeX
@article{liao2019identification,
  author = {Zhifang Liao and Ningwei Wang and Shengzong Liu and Yan Zhang and Hui Liu and Qi Zhang},
  title = {{Identification-Method Research for Open-Source Software Ecosystems}},
  journal = {Symmetry},
  year = {2019},
  doi = {10.3390/sym11020182},
}
Figure