Detecting Functionality-Specific Vulnerabilities via Retrieving Individual Functionality-Equivalent APIs in Open-Source Repositories

Tianyu Chen, Zeyu Wang, Lin Li, Ding Li, Zongyang Li, Xiaoning Chang, Pan Bian, Guangtai Liang, Qianxiang Wang, Tao Xie

TL;DR

This paper introduces APISS, a novel system that detects functionality-specific vulnerabilities in APIs by leveraging API documentation (doc strings) and signatures, rather than just their code bodies. APISS achieves this by first finding "functionality-equivalent" APIs and then adapting existing vulnerability exploit examples (PoCs) to these new APIs, leading to the discovery of numerous new vulnerabilities.

What Problem Does This Paper Address?

Software developers rely heavily on Application Programming Interfaces (APIs) from open-source repositories. However, these APIs can contain "functionality-specific vulnerabilities" – bugs tied to a particular function's purpose, like improper input validation in a file unzipping API. Existing methods for finding these vulnerabilities struggle because they often focus too much on the API's code implementation (its "body"). This is problematic because different APIs can perform the same function (they are "functionality-equivalent") but have vastly different code implementations, making it hard for traditional methods to find similar vulnerabilities across them. Furthermore, validating discovered vulnerabilities often requires complex "Proof-of-Concepts" (PoCs), which are time-consuming to create for new bugs.

Methods

APISS tackles these challenges with a two-pronged approach:

  1. Retrieving Functionality-Equivalent APIs: Instead of analyzing API code bodies, APISS uses Large Language Models (LLMs) to embed and compare API doc strings and signatures. The core idea is that APIs with similar functionalities will have similar documentation and signatures, even if their underlying code differs. This LLM-based embedding allows for more accurate and scalable identification of these equivalent APIs.
  2. Migrating Proof-of-Concepts (PoCs): Once functionality-equivalent APIs are identified, APISS semi-automatically adapts existing PoCs from known vulnerabilities to these new APIs. This significantly reduces the manual effort required to create new exploit examples for newly found bugs.

Datasets

The study utilizes a dataset of 180 widely used Java repositories. For training and evaluation, they collected source vulnerabilities from a Java vulnerability dataset (containing 1,359 vulnerabilities from 127 Java repositories) and their associated PoCs. Additionally, they gathered 42,612 candidate APIs from 180 randomly sampled Java repositories on GitHub, extracting their doc strings, signatures, and bodies.

Key Insights and Ideas

  • Documentation is Key: The most surprising and impactful insight is that API doc strings and signatures are far more indicative of functionality equivalence than the API's code body. This shifts the focus from complex code analysis to more accessible textual and structural information.
  • LLMs for Semantic Understanding: Leveraging LLMs for API embedding proves highly effective in capturing the semantic meaning of APIs, enabling better identification of functionality-equivalent counterparts across diverse implementations.
  • PoC Migration is Feasible: The observation that PoCs largely consist of input construction (consistent) and API invocation (variable but similar) makes semi-automatic migration a viable strategy, drastically reducing manual effort.
  • Signature is King (for Mapping): While doc strings are important, API signatures (including package and class names) appear to be the most critical feature for accurately mapping APIs to their functionality-equivalent counterparts.

Key Results

APISS demonstrates significant success:

  • Effectiveness in Vulnerability Detection: APISS detected 179 new vulnerabilities, with 60 receiving new CVE IDs. This is a substantial contribution to security practice.
  • API Mapping Accuracy: APISS achieved a Top-1 Accuracy of 0.81 for retrieving functionality-equivalent APIs, significantly outperforming the best baseline at 0.55.
  • Efficiency: Manual effort for migrating PoCs is reduced to under 10 minutes per vulnerability. The end-to-end runtime for analyzing a candidate API is less than 2 hours.
  • Reduced False Positives: Compared to baseline approaches, APISS exhibits a much higher precision score (0.40), indicating fewer false alarms.

Limitations and Future Work

  • Programming Language Focus: While APISS is designed to be generalizable, the current evaluation primarily focuses on Java vulnerabilities. Extending this to other languages is a natural next step.
  • Non-Functionality-Specific Vulnerabilities: APISS specifically targets functionality-specific vulnerabilities. The authors acknowledge that detecting non-functionality-specific vulnerabilities (like buffer overflows not tied to a particular function's purpose) is a separate challenge and a direction for future work.
  • Manual Effort for PoC Migration: While significantly reduced, some manual intervention is still required for PoC migration when the automatic parameter-matching algorithm fails.

Summary updated

Abstract

Functionality-specific vulnerabilities, which mainly occur in Application Programming Interfaces (APIs) with specific functionalities, are crucial for software developers to detect and avoid. When detecting individual functionality-specific vulnerabilities, the existing two categories of approaches are ineffective because they consider only the API bodies and are unable to handle diverse implementations of functionality-equivalent APIs. To effectively detect functionality-specific vulnerabilities, we propose APISS, the first approach to utilize API doc strings and signatures instead of API bodies. APISS first retrieves functionality-equivalent APIs for APIs with existing vulnerabilities and then migrates Proof-of-Concepts (PoCs) of the existing vulnerabilities for newly detected vulnerable APIs. To retrieve functionality-equivalent APIs, we leverage a Large Language Model for API embedding to improve the accuracy and address the effectiveness and scalability issues suffered by the existing approaches. To migrate PoCs of the existing vulnerabilities for newly detected vulnerable APIs, we design a semi-automatic schema to substantially reduce manual costs. We conduct a comprehensive evaluation to empirically compare APISS with four state-of-the-art approaches of detecting vulnerabilities and two state-of-the-art approaches of retrieving functionality-equivalent APIs. The evaluation subjects include 180 widely used Java repositories using 10 existing vulnerabilities, along with their PoCs. The results show that APISS effectively retrieves functionality-equivalent APIs, achieving a Top-1 Accuracy of 0.81 while the best of the baselines under comparison achieves only 0.55. APISS is highly efficient: the manual costs are within 10 minutes per vulnerability and the end-to-end runtime overhead of testing one candidate API is less than 2 hours. APISS detects 179 new vulnerabilities and receives 60 new CVE IDs, bringing high value to security practice.
BibTeX
@inproceedings{chen2025detecting,
  author = {Tianyu Chen and Zeyu Wang and Lin Li and Ding Li and Zongyang Li and Xiaoning Chang and Pan Bian and Guangtai Liang and Qianxiang Wang and Tao Xie},
  title = {{Detecting Functionality-Specific Vulnerabilities via Retrieving Individual Functionality-Equivalent APIs in Open-Source Repositories}},
  booktitle = {ECOOP},
  year = {2025},
  doi = {10.4230/lipics.ecoop.2025.6},
}
Figure