TL;DR
PatchFuzz is an automated framework designed to find vulnerabilities in JavaScript (JS) engines by focusing on "hot code"—the areas most frequently modified by security patches. By automatically identifying security-related commits, extracting and "repairing" their Proof-of-Concept (PoC) triggers, and using selective instrumentation, the tool uncovered 54 new bugs in major engines like V8 and JavaScriptCore, earning $62,500 in bounties.
What Problem Does This Paper Address?
JavaScript engines are the heart of modern web browsers, making them high-value targets for attackers. While fuzzing (feeding random inputs to a program to find crashes) is a standard testing technique, "patch fuzzing" is a more surgical approach. It targets newly patched code under the assumption that security fixes are often incomplete or inadvertently introduce new "follow-up" vulnerabilities.
However, patch fuzzing for JS engines faces three major hurdles:
- Data Collection: Manually finding security-related commits in massive git histories is tedious, especially since developers often "silently" fix bugs without assigning CVE IDs.
- Seed Quality: PoCs attached to commits often don't run out-of-the-box because they rely on internal engine test-runners or specific environment variables.
- Efficiency: Standard fuzzers instrument the entire engine, wasting computational power on stable, "cold" code rather than the volatile, "hot" patched areas.
Methods
The authors propose PatchFuzz, an end-to-end pipeline that automates the entire patch-fuzzing lifecycle:
- Rule-Based Recognition: Instead of relying on generic machine learning models (which the authors show perform poorly on JS-specific data), PatchFuzz uses heuristic rules—looking for security keywords, bug report links, and the ratio of conditional statements—to identify security patches in git commits.
- PoC Repair: The system extracts JS files from these commits and automatically fixes common execution errors (e.g., replacing engine-specific
$vmcalls with standardprintstatements) to turn them into usable fuzzing seeds. - Selective Instrumentation: PatchFuzz uses call graphs to identify and instrument only the functions directly modified by a patch and their immediate neighbors. This focuses the fuzzer's "feedback" loop on the most suspicious code paths.
- Dual-Intensity Mutation: To avoid "smashing" the delicate logic of a PoC, PatchFuzz uses a hybrid strategy: 80% low-intensity mutations (expression-level changes) to stay near the original buggy path, and 20% high-intensity mutations (statement-level changes) to explore further.
Datasets
The study evaluates PatchFuzz against six major open-source JS engines: JavaScriptCore (Safari), V8 (Chrome), SpiderMonkey (Firefox), ChakraCore (Edge), JerryScript, and QuickJS.
The authors introduced:
- A manually labeled dataset of 600 git commits (used to benchmark the patch recognizer).
- A massive collection of 17,477 extracted PoCs, which they claim is the largest high-quality security PoC dataset for JS engines to date.
Key Insights and Ideas
- The "Hot Code" Reality: The authors found that roughly 35.8% of source files in JS engines are "hot," meaning they are patched repeatedly. This confirms that focusing fuzzing resources on previously buggy areas is a highly effective strategy.
- ML isn't a Silver Bullet: State-of-the-art vulnerability detection models like VulCurator failed significantly when applied to JS engines (achieving an F1 score of only 0.09). Domain-specific heuristics proved much more reliable for identifying security commits.
- Preserving PoC Integrity: A key "aha!" moment was realizing that aggressive mutation often destroys the JIT (Just-In-Time) compiler optimizations that trigger specific bugs. By using a conservative mutation strategy, PatchFuzz keeps the "penetrating power" of the original PoC intact while still exploring nearby code.
Key Results
- Bug Discovery: PatchFuzz found 54 unique bugs across the six engines. 25 of these were assigned CVEs, and the researchers received $62,500 in bug bounties.
- Superior Coverage: In head-to-head tests, PatchFuzz achieved approximately 30% higher code coverage than top-tier baselines like Superion and Fuzzilli.
- PoC Usability: The automated repair tool reduced the grammar error rate of extracted PoCs from an average of 27.5% down to 8.4%, significantly increasing the pool of viable fuzzing seeds.
Limitations and Future Work
While highly effective, PatchFuzz still has areas for improvement:
- Residual Errors: About 8.4% of PoCs still fail to execute after automated repair, often due to complex environmental dependencies that require deeper semantic analysis to fix.
- Heuristic Maintenance: The rule-based recognizer depends on developer commit habits; if those habits change significantly, the rules may need manual updates.
- Expansion: The authors suggest that this "patch-centric" sustainable fuzzing model could be adapted for other large-scale software projects beyond JavaScript engines.
Data Availability: The authors have open-sourced PatchFuzz on GitHub at https://github.com/PatchFuzz/patchfuzz.
Summary updated