Data Correction and Evolution Analysis of the ProgrammableWeb Service Ecosystem

Mingyi Liu, Zhiying Tu, Yeqi Zhu, Xiaofei Xu, Zhongjie Wang, Quan Z. Sheng

Abstract

The evolution analysis on Web service ecosystems has become a critical problem as the frequency of service changes on the Internet increases rapidly. Developers need to understand these evolution patterns to assist in their decision-making on service selection. ProgrammableWeb is a popular Web service ecosystem on which several evolution analyses have been conducted in the literature. However, the existing studies have ignored the quality issues of the ProgrammableWeb dataset and the issue of service obsolescence. In this study, we first report the quality issues identified in the ProgrammableWeb dataset from our empirical study. Then, we propose a novel method to correct the relevant evolution analysis data by estimating the life cycle of application programming interfaces (APIs) and mashups. We also reveal how to use three different dynamic network models in the service ecosystem evolution analysis based on the corrected ProgrammableWeb dataset. Our experimental experience iterates the quality issues of the original ProgrammableWeb and highlights several research opportunities.

Introduction

For over a decade, researchers have relied on the ProgrammableWeb (PW) dataset to study the "API economy"—the intricate network where web services (APIs) are integrated into applications (mashups). However, this paper provides a necessary "reality check" for the community by uncovering significant data quality issues in this industry-standard repository.

Most existing evolution analyses assume the PW data is high-quality and ignore service obsolescence. In reality, many APIs listed as "active" are dead links, and the "death dates" provided by PW are often administrative artifacts rather than actual timestamps of service failure. This leads to a "blindly optimistic" view of the ecosystem's health, resulting in inaccurate service recommendations and flawed models of network growth.

Methodology: A Pipeline for Data Correction

To address these flaws, the authors developed a multi-stage pipeline encompassing identification, correction, and re-analysis of the service ecosystem.

The research framework for data quality analysis and evolution analysis, moving from raw data to corrected evolution models.

The process begins with a rigorous Data Auditing phase. Using automated network request testing and manual inspection by expert coders, the team identified three primary issues: untrustworthy death times, incorrect API statuses, and incorrect mashup compositions.

Because the exact death date for many services is unknown, the authors implemented a Probabilistic Correction method. They estimated service lifespans using a Normal Distribution-based approach, validated against a manually verified sample of 200 APIs. Finally, they applied Dynamic Network Modeling at three levels: Mashup-API (bipartite graph), API-API (co-occurrence network), and Category-Category (inter-domain interaction).

Uncovering the "Deathpool" Mess

The empirical study revealed startling discrepancies in the original dataset. A major finding was that 85.2% of unavailable APIs were incorrectly marked as "active." Furthermore, the "death times" recorded in the database were often clustered around specific dates due to administrative batch processing.

Distribution of API death times in the original dataset, showing administrative artifacts like the massive spike in April 2014.

As shown in Figure 2, thousands of APIs were marked "dead" on a single day in April 2014. Such artifacts severely distort evolution analysis if not corrected. By applying their correction methods to the raw dataset (23,881 APIs and 7,973 mashups), the authors were able to generate a more realistic "Corrected ProgrammableWeb Dataset," which they have released on GitHub.

Key Insights: An Ecosystem in Decline

The re-analysis of the corrected data yielded several insights that contradict previous "explosive growth" narratives:

  • The Ecosystem is Shrinking: The number of active APIs and mashups actually peaked around 2014 and has been declining since. This likely reflects a shift from Web-based mashups to mobile applications.
  • Complexity Plateau: Since 2012, the complexity of mashups (number of APIs invoked) has stalled, with most mashups typically using only 2-3 APIs.
  • Popularity ≠ Survival: Using frequently co-occurring API pairs (e.g., Twitter + Google Maps) does not guarantee mashup survival; some high-frequency pairs have survival rates as low as 20%.
  • Declining Diversity and Connectivity: The diversity of service categories began to decline after 2017. Additionally, the "largest connected component" in the API network has shrunk since 2013, indicating a more fragmented ecosystem.

Conclusion and Future Directions

While the proposed correction method is probabilistic and cannot guarantee the exact death date for every individual API, it successfully restores the overall distribution of the ecosystem for more accurate macro-level analysis.

This research highlights the urgent need for the community to look beyond ProgrammableWeb to capture the modern, fragmented landscape of mobile and enterprise APIs. Future work will focus on building more sophisticated regression models to predict service lifespans as more labeled data becomes available.

Summary updated

Abstract

The evolution analysis on Web service ecosystems has become a critical problem as the frequency of service changes on the Internet increases rapidly. Developers need to understand these evolution patterns to assist in their decision-making on service selection. ProgrammableWeb is a popular Web service ecosystem on which several evolution analyses have been conducted in the literature. However, the existing studies have ignored the quality issues of the ProgrammableWeb dataset and the issue of service obsolescence. In this study, we first report the quality issues identified in the ProgrammableWeb dataset from our empirical study. Then, we propose a novel method to correct the relevant evolution analysis data by estimating the life cycle of application programming interfaces (APIs) and mashups. We also reveal how to use three different dynamic network models in the service ecosystem evolution analysis based on the corrected ProgrammableWeb dataset. Our experimental experience iterates the quality issues of the original ProgrammableWeb and highlights several research opportunities.
BibTeX
@article{liu2021data,
  author = {Mingyi Liu and Zhiying Tu and Yeqi Zhu and Xiaofei Xu and Zhongjie Wang and Quan Z. Sheng},
  title = {{Data Correction and Evolution Analysis of the ProgrammableWeb Service Ecosystem}},
  journal = {Journal of Systems and Software},
  year = {2021},
  url = {https://openreview.net/forum?id=ATanVB1J6T},
}

Loading related content...

Figure