A Quantitative Study of Social Organisation in Open Source Software Communities

Marcelo Serrano Zanetti, Emre Sarigöl, Ingo Scholtes, Claudio Juan Tessone, Frank Schweitzer

TL;DR

This paper quantitatively analyzes the social organization of 14 major open-source software (OSS) communities by examining their collaboration networks over time. By applying network analysis metrics, the authors reveal significant differences in community structure, cohesion, and resilience, suggesting these can be used as indicators of project health and risk.

What Problem Does This Paper Address?

The success of open-source software (OSS) projects hinges on the collective efforts of a large, voluntary community. While project success is often linked to community size, the structure and dynamics of these social collaborations are equally crucial but harder to quantify. This paper addresses the challenge of understanding and measuring the social organization within OSS communities, moving beyond simple metrics like contributor count to explore how members interact, share responsibilities, and how resilient the community is to member turnover. The authors aim to develop quantitative indicators that can signal the health and potential risks of these vital communities.

Methods

The core of this research lies in applying network analysis to model community interactions. The authors construct dynamic collaboration networks from the historical data of 14 OSS projects. Specifically, they extract interactions from Bugzilla bug-tracker installations, focusing on changes to the "ASSIGNEE" (who is responsible for fixing a bug) and "CC" (who is notified about updates) fields. These interactions are treated as time-stamped edges between users.

To handle the vast amount of data and capture evolving dynamics, they employ a 30-day sliding time window to generate monthly network snapshots. To ensure the validity of their network analysis measures, they focus on the largest connected component (LCC) of each monthly network.

They then apply three key network measures:

  • Closeness Centrality: Measures how quickly a node (user) can reach all other nodes, indicating how distributed responsibilities and communication are.
  • Clustering Coefficient: Quantifies how closely knit the community is, measuring the tendency for nodes to form tightly connected groups.
  • Degree Assortativity: Examines whether nodes with similar degrees (number of connections) tend to connect to each other (assortative) or if nodes with different degrees connect (disassortative).

Datasets

The study utilizes a large-scale dataset collected from the Bugzilla bug-tracker installations of 14 major open-source software projects. The dataset covers the full, fine-grained history of user collaborations for these projects, with some histories extending over ten years. The paper provides an aggregated overview of these projects in Table 1, including metrics like the number of bugs, updates, and the time period covered.

Key Insights and Ideas

  • Beyond Size: Community size is not the sole determinant of success; the structure of collaboration is critical.
  • Dynamic Evolution: OSS communities are not static. Their social organization evolves over time, and these changes can be tracked and analyzed.
  • Divergent Social Architectures: Even projects of similar size and age can exhibit vastly different social organization patterns, as evidenced by the contrasting clustering coefficients of ECLIPSE and GENTOO.
  • Potential Risk Indicators: Phases of high closeness centralization coupled with low clustering (as seen in GENTOO) might indicate a lack of social cohesion, potentially posing a risk to the project's sustainability.
  • Assortativity as a Differentiator: Degree assortativity proves to be a useful metric for further distinguishing between the social organizations of different OSS communities.

Key Results

The paper presents a comparative analysis of the 14 OSS communities, highlighting differences in their social organization.

  • Cohesiveness: The clustering coefficient varied significantly across projects. For instance, the ECLIPSE community showed a clustering coefficient approximately ten times higher than GENTOO between 2006 and 2008, despite similar LCC sizes.
  • Distribution of Responsibilities: Closeness centralization revealed distinct patterns. The GENTOO community exhibited a notable plateau in closeness centralization between 2006 and 2008, suggesting a potential bottleneck or reliance on a few central members.
  • Interaction Preferences: Degree assortativity also showed project-specific trends. ECLIPSE tended towards assortative structures, while KDE and GENTOO leaned towards disassortativity.
  • LCC Stability: The fraction of nodes within the largest connected component (LCC) remained relatively stable for most projects (between 0.7 and 1), indicating that the core community structure is generally consistent.

Limitations and Future Work

The authors acknowledge that their results are preliminary. They plan to extend their work by incorporating:

  • Spectral measures like algebraic connectivity.
  • Inequality measures such as the Gini index.
  • Community performance indicators (e.g., response times, bug fixing times) for a more comprehensive analysis.

The ultimate goal is to develop multi-dimensional indicators for social and technical organization that correlate with performance, aiding in investment decisions and risk management for OSS projects. They also envision these indicators being integrated into community platforms like Bugzilla to help project managers proactively identify and mitigate risks.

Summary updated

Abstract

The success of open source projects crucially depends on the voluntary contributions of a sufficiently large community of users. Apart from the mere size of the community, interesting questions arise when looking at the evolution of structural features of collaborations between community members. In this article, we discuss several network analytic proxies that can be used to quantify different aspects of the social organisation in social collaboration networks. We particularly focus on measures that can be related to the cohesiveness of the communities, the distribution of responsibilities and the resilience against turnover of community members. We present a comparative analysis on a large-scale dataset that covers the full history of collaborations between users of $14$ major open source software communities. Our analysis covers both aggregate and time-evolving measures and highlights differences in the social organisation across communities. We argue that our results are a promising step towards the definition of suitable, potentially multi-dimensional, resilience and risk indicators for open source software communities.
BibTeX
@inproceedings{zanetti2012quantitative,
  author = {Marcelo Serrano Zanetti and Emre Sarigöl and Ingo Scholtes and Claudio Juan Tessone and Frank Schweitzer},
  title = {{A Quantitative Study of Social Organisation in Open Source Software Communities}},
  booktitle = {ICCSW},
  year = {2012},
  doi = {10.4230/oasics.iccsw.2012.116},
}

Loading related content...

Figure