Community Vital Signs: Measuring Wikipedia Communities’ Sustainable Growth and Renewal

Marc Miquel-Ribé, Cristian Consonni, David LaniadoSustainability 2022 · DOI

TL;DR

This paper introduces "Vital Signs," a novel set of six language-independent indicators to measure the health and sustainability of Wikipedia communities. By analyzing editor activity patterns, the study reveals that many Wikipedia language editions are not in decline as previously thought, but are actively renewing their contributor base, though specific roles like administrators show a concerning lack of renewal.

What Problem Does This Paper Address?

Wikipedia, a monumental collaborative project, has seen a massive influx of volunteer contributors. However, since around 2007, research has indicated a decline in the number of active editors, particularly in larger language editions. This has raised concerns about the long-term sustainability and health of these online communities. Existing metrics primarily focus on raw editor numbers, failing to capture the nuanced dynamics of community growth, renewal, and the health of specialized contributor groups. This paper aims to fill this gap by developing a comprehensive framework to assess and monitor community vitality beyond simple editor counts.

Methods

The authors propose six "Vital Signs" indicators, drawing an analogy to medical vital signs for assessing community health. These indicators are derived from digital traces left by editors on Wikipedia and are designed to be language-independent. The Vital Signs are categorized into two groups:

  • General Editor Population:
    • Retention: Measures the capacity of a community to keep new editors engaged after their initial contributions (defined as editing at least 60 days after their first edit).
  • Stability: Assesses the persistence of active editors over time, looking at how many months editors remain active consecutively.
  • Balance: Examines the composition of the active editor base across different generations (defined by the year of their first edit), aiming for a mix of experienced and new contributors.
  • Specific Community Functions:
    • Special Functions: Focuses on technical editors (those working on templates and MediaWiki namespaces) and coordinators (those active in Wikipedia and Help namespaces), assessing their renewal and engagement.
  • Administrators: Analyzes the granting of admin rights, looking at the generation of new administrators and their proportion relative to active editors.
  • Global Participation: Measures participation in Meta-wiki (the Wikimedia movement's coordination platform) and the proportion of editors whose primary contribution is to a specific language edition versus those who contribute to multiple editions.

The methodology involved an iterative process of exploration (literature review, data analysis), design (indicator definition, visualization), and validation (focus groups with community members and affiliates). Time-series clustering (k-means with dynamic time warping) was used to identify growth/stagnation/decline patterns across 308 Wikipedia language editions.

Datasets

The study utilized data from Wikimedia Foundation dumps, specifically the Mediawiki History dump, covering interactions up to November 2021. The code and resulting databases are made publicly available.

Key Insights and Ideas

  • Not all Wikipedias are declining: Contrary to common assumptions, the study found that about half of the 50 largest Wikipedia language communities are not stagnating or declining. Some, like Arabic and Chinese, are still growing significantly.
  • Renewal is happening, but not everywhere: While many communities are renewing their active editor base, there's a notable lack of renewal in specialized roles like administrators and technical editors, often dominated by older generations of editors.
  • The "Vital Signs" framework offers a nuanced view: Moving beyond simple editor counts, these indicators provide a more detailed understanding of community health, allowing for proactive identification of risks and opportunities.
  • Affiliates are key to driving change: The validation phase highlighted that Wikimedia affiliates, being community-based organizations, are well-positioned to lead initiatives for improving community health and growth.

Key Results

  • Growth/Stagnation/Decline Patterns: Analysis of 308 language editions revealed six distinct temporal patterns of editor activity, with some showing sustained growth, others stagnation, and some decline.
  • Retention is declining: Across all eight selected language editions analyzed, the retention rate of new editors has been declining over the last 10 years, a trend considered worrying.
  • Stability varies by community maturity: More established communities show higher editor stability (editors remaining active for longer consecutive months), while younger projects exhibit more volatility.
  • Balance shows generational dominance: In many communities, the group of "very active editors" is dominated by older generations, suggesting a potential barrier for newer editors to reach these productive roles.
  • Specialized roles lack renewal: Both technical editors and coordinators, crucial for community functions, show a strong reliance on older generations, indicating a need for targeted recruitment and mentorship.
  • Admin roles are concentrated: Administrator positions are heavily dominated by older generations, with a significant lack of new administrators being appointed in recent years, raising concerns about power concentration.
  • Global participation is generally low: Participation in Meta-wiki is typically around 1% of active editors for large communities, though it can be higher for smaller ones.

Limitations and Future Work

The authors acknowledge that the study focused on "very active editors" for balance analysis, which is a subset of the entire community. They also note that the validation process relied on qualitative feedback from focus groups, and a more extensive quantitative survey could provide further insights.

Future work includes:

  • Developing an interactive dashboard to provide regular updates of the Vital Signs for all Wikipedia language editions.
  • Further analyzing the relationship between active community size, registered editors, retained editors, and those who abandon the project.
  • Investigating the causes of retention decline and exploring other dimensions of the editing experience, such as discussion patterns and editor satisfaction with tools.
  • Potentially incorporating additional Vital Signs, such as drop-off rates and administrative actions.
  • Exploring the concentration of functions and skills among editors.

Summary updated

Abstract

Wikipedia is an undeniably successful project, with unprecedented numbers of online volunteer contributors. After 2007, researchers started to observe that the number of active editors for the largest Wikipedias declined after rapid initial growth. Years after those announcements, researchers and community activists still need to understand how to measure community health. In this paper, we study patterns of growth, decline and stagnation, and we propose the creation of 6 sets of language-independent indicators that we call “Vital Signs”. Three focus on the general population of active editors creating content: retention, stability, and balance; the other three are related to specific community functions: specialists, administrators, and global community participation. We borrow the analogy from the medical field, as these indicators represent a first step in defining the health status of a community; they can constitute a valuable reference point to foresee and prevent future risks. We present our analysis for eight Wikipedia language editions, and we show that communities are renewing their productive force even with stagnating absolute numbers; we observe a general lack of renewal in positions related to special functions or administratorship. Finally, we evaluate our framework by discussing these indicators with Wikimedia affiliates to support them in promoting the necessary changes to grow the communities.
BibTeX
@inproceedings{miquelribe2022community,
  author = {Marc Miquel-Ribé and Cristian Consonni and David Laniado},
  title = {{Community Vital Signs: Measuring Wikipedia Communities’ Sustainable Growth and Renewal}},
  booktitle = {Sustainability},
  year = {2022},
  doi = {10.3390/su14084705},
}

Loading related content...

Figure