TL;DR
This paper introduces APISS, a novel system that detects functionality-specific vulnerabilities in APIs by leveraging API documentation (doc strings) and signatures, rather than just their code bodies. APISS achieves this by first finding "functionality-equivalent" APIs and then adapting existing vulnerability exploit examples (PoCs) to these new APIs, leading to the discovery of numerous new vulnerabilities.
What Problem Does This Paper Address?
Software developers rely heavily on Application Programming Interfaces (APIs) from open-source repositories. However, these APIs can contain "functionality-specific vulnerabilities" – bugs tied to a particular function's purpose, like improper input validation in a file unzipping API. Existing methods for finding these vulnerabilities struggle because they often focus too much on the API's code implementation (its "body"). This is problematic because different APIs can perform the same function (they are "functionality-equivalent") but have vastly different code implementations, making it hard for traditional methods to find similar vulnerabilities across them. Furthermore, validating discovered vulnerabilities often requires complex "Proof-of-Concepts" (PoCs), which are time-consuming to create for new bugs.
Methods
APISS tackles these challenges with a two-pronged approach:
- Retrieving Functionality-Equivalent APIs: Instead of analyzing API code bodies, APISS uses Large Language Models (LLMs) to embed and compare API doc strings and signatures. The core idea is that APIs with similar functionalities will have similar documentation and signatures, even if their underlying code differs. This LLM-based embedding allows for more accurate and scalable identification of these equivalent APIs.
- Migrating Proof-of-Concepts (PoCs): Once functionality-equivalent APIs are identified, APISS semi-automatically adapts existing PoCs from known vulnerabilities to these new APIs. This significantly reduces the manual effort required to create new exploit examples for newly found bugs.
Datasets
The study utilizes a dataset of 180 widely used Java repositories. For training and evaluation, they collected source vulnerabilities from a Java vulnerability dataset (containing 1,359 vulnerabilities from 127 Java repositories) and their associated PoCs. Additionally, they gathered 42,612 candidate APIs from 180 randomly sampled Java repositories on GitHub, extracting their doc strings, signatures, and bodies.
Key Insights and Ideas
- Documentation is Key: The most surprising and impactful insight is that API doc strings and signatures are far more indicative of functionality equivalence than the API's code body. This shifts the focus from complex code analysis to more accessible textual and structural information.
- LLMs for Semantic Understanding: Leveraging LLMs for API embedding proves highly effective in capturing the semantic meaning of APIs, enabling better identification of functionality-equivalent counterparts across diverse implementations.
- PoC Migration is Feasible: The observation that PoCs largely consist of input construction (consistent) and API invocation (variable but similar) makes semi-automatic migration a viable strategy, drastically reducing manual effort.
- Signature is King (for Mapping): While doc strings are important, API signatures (including package and class names) appear to be the most critical feature for accurately mapping APIs to their functionality-equivalent counterparts.
Key Results
APISS demonstrates significant success:
- Effectiveness in Vulnerability Detection: APISS detected 179 new vulnerabilities, with 60 receiving new CVE IDs. This is a substantial contribution to security practice.
- API Mapping Accuracy: APISS achieved a Top-1 Accuracy of 0.81 for retrieving functionality-equivalent APIs, significantly outperforming the best baseline at 0.55.
- Efficiency: Manual effort for migrating PoCs is reduced to under 10 minutes per vulnerability. The end-to-end runtime for analyzing a candidate API is less than 2 hours.
- Reduced False Positives: Compared to baseline approaches, APISS exhibits a much higher precision score (0.40), indicating fewer false alarms.
Limitations and Future Work
- Programming Language Focus: While APISS is designed to be generalizable, the current evaluation primarily focuses on Java vulnerabilities. Extending this to other languages is a natural next step.
- Non-Functionality-Specific Vulnerabilities: APISS specifically targets functionality-specific vulnerabilities. The authors acknowledge that detecting non-functionality-specific vulnerabilities (like buffer overflows not tied to a particular function's purpose) is a separate challenge and a direction for future work.
- Manual Effort for PoC Migration: While significantly reduced, some manual intervention is still required for PoC migration when the automatic parameter-matching algorithm fails.
Summary updated