TL;DR
This paper provides a much-needed "reality check" for the Web service research community by uncovering significant data quality issues in the industry-standard ProgrammableWeb dataset. By developing a novel probabilistic method to correct service "death times" and active statuses, the authors reveal that the API ecosystem is actually in a state of decline rather than the "explosive growth" reported by previous studies.
What Problem Does This Paper Address?
For over a decade, researchers have used the ProgrammableWeb (PW) dataset to study the "API economy"—how web services (APIs) are composed into applications (mashups). However, most existing evolution analyses share a fatal flaw: they assume the data is high-quality and ignore service obsolescence.
In reality, many APIs listed as "active" are dead links, and the "death dates" provided by PW are often administrative artifacts (e.g., thousands of APIs being marked "dead" on a single day in April 2014) rather than actual timestamps of service failure. This leads to a "blindly optimistic" view of the ecosystem's health, resulting in inaccurate service recommendations and flawed models of how these networks grow.
Methods
The authors tackle this through a multi-stage pipeline of identification, correction, and re-analysis:
- Data Auditing: They used automated network request testing (checking if API endpoints actually respond) and manual inspection by five expert coders to identify three main issues: untrustworthy death times, incorrect API statuses, and incorrect mashup compositions.
- Probabilistic Correction: Since the exact death date for many services is unknown, the authors estimated service lifespans using a Normal Distribution-based method. They validated this distribution against a manually verified sample of 200 APIs using Z-statistics to ensure the corrected "deathpool" reflected a realistic overall distribution.
- Dynamic Network Modeling: They modeled the ecosystem at three levels:
- Mashup-API (M-A): A bipartite graph showing which mashups invoke which APIs.
- API-API (A-A): A network where APIs are linked if they are used together in the same mashup.
- Category-Category (C-C): A high-level view of how different service domains (e.g., Social, Mapping, Search) interact.
Datasets
- ProgrammableWeb (Original): The raw dataset containing 23,881 APIs and 7,973 mashups collected as of January 2021.
- Corrected ProgrammableWeb Dataset: The authors have released their cleaned version, which includes corrected active statuses and estimated active timeframes, on GitHub.
Key Insights and Ideas
- The "Deathpool" is a Mess: The authors found that 85.2% of unavailable APIs were incorrectly marked as "active" in the original dataset. Many APIs were "born" after they were supposedly "dead" due to administrative errors in the PW database.
- The Ecosystem is Shrinking: Contrary to the narrative of "explosive growth," the number of active APIs and mashups actually peaked around 2014 and has been declining since. The authors suggest this might be due to the shift from Web-based mashups to mobile apps.
- Complexity has Plateaued: Since 2012, the complexity of mashups (the number of APIs they invoke) has not increased. Most mashups remain simple, typically using only 2-3 APIs.
- Popularity $\neq$ Survival: Interestingly, the study found that using frequently co-occurring API pairs (like Twitter + Google Maps) does not actually increase a mashup's chance of survival. In fact, some high-frequency pairs have survival rates as low as 20%.
Key Results
- Active Status: Only 44.7% of the APIs listed in the original dataset are truly active.
- Network Laws: While previous studies claimed the API network follows a strict Power Law distribution (where a few "super-nodes" dominate), the authors' statistical $p$-value analysis suggests this hypothesis is less stable than previously thought.
- Diversity Decline: The diversity of the ecosystem (measured by the number of active service categories) entered a plateau and began to decline rapidly after 2017.
- Connectivity: The size of the "largest connected component" in the API network has continued to shrink since 2013, indicating a more fragmented and less collaborative ecosystem.
Limitations and Future Work
The authors acknowledge that their correction method is probabilistic. While it accurately restores the overall distribution of the ecosystem, it cannot guarantee that the estimated death date for a specific individual API is 100% correct.
Future work could involve building more sophisticated regression models to predict service lifespans if more labeled data becomes available. Additionally, the authors suggest that the research community needs to look beyond ProgrammableWeb to capture the modern, more fragmented landscape of mobile and enterprise APIs that may not be indexed on public repositories.
Summary updated