Online division of labour: emergent structures in Open Source Software

María J. Palazzi, Jordi Cabot, Javier Luis Izquierdo, Albert Solé‐Ribalta, Javier Borge‐Holthoefer

While Open Source Software (OSS) platforms like GitHub offer a virtually boundless environment for collaboration, human cognitive limits still dictate how work gets done. By analyzing developer-file interaction networks, researchers found that contributors naturally self-organize into specialized, bounded communities of roughly 100-200 people—mirroring the famous Dunbar number. Furthermore, as projects mature, these communities develop internal hierarchies, revealing a predictable evolution of the division of labor in decentralized online environments.

What Problem Does This Paper Address?

Platforms like GitHub provide a decentralized, asynchronous, and remote framework where an unlimited number of developers can theoretically collaborate on an unlimited number of files. However, human beings have well-documented biological and cognitive limits regarding how many social connections they can maintain (often referred to as the Dunbar number, typically estimated between 100 and 250).

The authors investigate a fascinating tension: How do massive, distributed OSS communities self-organize to overcome these cognitive limits without top-down corporate management? Specifically, they look for structural signatures of the "division of labor" to see if developers naturally compartmentalize into specialized sub-groups, and whether those sub-groups develop internal hierarchies to maintain efficiency as the project scales.

Methods

The researchers modeled OSS projects as bipartite unweighted networks. In these graphs, there are two types of nodes: contributors and files. An edge exists if a contributor has edited a specific file.

To understand the architecture of these networks, they measured three distinct mesoscale structural patterns:

  1. Nestedness ($\mathcal{N}$): A hierarchical structure where specialists (developers who edit few files) work on a strict subset of the files edited by generalists (developers who edit many files).
  2. Modularity ($\mathcal{Q}$): The tendency of the network to split into distinct, well-connected compartments (communities of developers focusing on specific groups of files).
  3. In-block nestedness ($\mathcal{I}$): A hybrid structure where the network is modular, but the internal structure of each module is nested.

To find these structures, the authors used an extremal optimization algorithm combined with a Kernighan-Lin refinement procedure to maximize modularity and in-block nestedness. They then mapped the projects onto a ternary plot (a simplex) to visualize which of the three architectures dominated each project based on its evolutionary stage.

Datasets

The study analyzed 65 of the top 100 most popular public OSS projects on GitHub (measured by stars as of 2016).

  • Data Cleaning: The authors did heavy preprocessing using the Gitana tool and the GitHub API to resolve Git's notorious "clashing and duplicity" problem (where one developer uses multiple emails/names, or multiple developers use the same common name).
  • Scale: The final dataset encompassed 48,015 contributors, 668,283 files, and 912,766 commits. Project network sizes (contributors + files) ranged from 82 to 13,382 nodes.

Key Insights and Ideas

  • The Dunbar Number holds online: Even in massive projects, a developer's "implicit average degree" (the number of co-contributors they indirectly interact with by editing the same files) remains flat at around 70 people.
  • Communities have a hard size limit: When the algorithm partitions the projects into modules, the maximum size of these sub-communities naturally caps out. Instead of scaling infinitely with the project, sub-groups max out between 100 and 200 contributors, perfectly echoing Dunbar's cognitive limits.
  • Maturity breeds internal hierarchy: The ternary plot revealed a lifecycle to OSS architecture. Newer or less active projects tend to be purely modular (fragmented teams working on specific parts). However, highly mature projects (those with the most commits) evolve an "in-block nested" structure. This means that within a specialized module, a clear hierarchy of generalist "core" developers and specialist "drive-by" developers emerges.

Key Results

  • High structural organization: 90% of the analyzed projects exhibited strong modularity ($\mathcal{Q}$) or in-block nestedness ($\mathcal{I}$) scores above 0.4, indicating that division of labor is a nearly universal feature of successful OSS projects.
  • Log-normal community sizes: The distribution of the largest community sizes across all projects fits a log-normal distribution centered tightly around 100 contributors.
  • Scale independence: For projects with a total network size greater than 2,000 nodes, the maximum community size stabilizes at ~100-200 contributors. This proves that workload compartmentalization is independent of the total project scale; a project doesn't get larger communities, it just gets more of them.

Limitations and Future Work

  • Selection Bias: The study only looked at highly successful, popular GitHub projects. Failed, abandoned, or average projects might exhibit entirely different structural patterns (or a lack thereof).
  • Static Snapshots: The bipartite networks aggregate the entire history of a project into a single static matrix, ignoring the temporal dynamics of how developers join, contribute, and leave.
  • Future Directions: The authors suggest analyzing time-resolved datasets to watch these modular and nested structures form dynamically. They also propose building generative network models with specific dynamical rules for both files and contributors to simulate how OSS projects grow and evolve over time.

Summary updated

Abstract

The development Open Source Software fundamentally depends on the participation and commitment of volunteer developers to progress on a particular task. Several works have presented strategies to increase the on-boarding and engagement of new contributors, but little is known on how these diverse groups of developers self-organise to work together. To understand this, one must consider that, on one hand, platforms like GitHub provide a virtually unlimited development framework: any number of actors can potentially join to contribute in a decentralised, distributed, remote, and asynchronous manner. On the other, however, it seems reasonable that some sort of hierarchy and division of labour must be in place to meet human biological and cognitive limits, and also to achieve some level of efficiency. These latter features (hierarchy and division of labour) should translate into detectable structural arrangements when projects are represented as developer-file bipartite networks. Thus, in this paper we analyse a set of popular open source projects from GitHub, placing the accent on three key properties: nestedness, modularity and in-block nestedness -which typify the emergence of heterogeneities among contributors, the emergence of subgroups of developers working on specific subgroups of files, and a mixture of the two previous, respectively. These analyses show that indeed projects evolve into internally organised blocks. Furthermore, the distribution of sizes of such blocks is bounded, connecting our results to the celebrated Dunbar number both in off- and on-line environments. Our conclusions create a link between bio-cognitive constraints, group formation and online working environments, opening up a rich scenario for future research on (online) work team assembly (e.g. size, composition, and formation). From a complex network perspective, our results pave the way for the study of time-resolved datasets, and the design of suitable models that can mimic the growth and evolution of OSS projects.
BibTeX
@inproceedings{palazzi2020online,
  author = {María J. Palazzi and Jordi Cabot and Javier Luis Izquierdo and Albert Solé‐Ribalta and Javier Borge‐Holthoefer},
  title = {{Online division of labour: emergent structures in Open Source Software}},
  booktitle = {LA},
  year = {2020},
}

Loading related content...

Figure