FalseCrashReducer: Mitigating False Positive Crashes in OSS-Fuzz-Gen Using Agentic AI

Paschal C. Amusuo, Dongge Liu, Ricardo Andrés Calvo Méndez, Jonathan Metzman, Oliver Chang, James C. DavisarXiv 2025 · DOI · arXiv

Introduction

Fuzz testing is a cornerstone of modern software security, yet its automation remains a significant challenge. While tools like Google’s OSS-Fuzz-Gen leverage Large Language Models (LLMs) to automate the creation of "fuzz drivers"—the glue code required to test individual functions—they frequently encounter a "bottom-up" testing hurdle. By testing functions in isolation, these tools often generate execution states that are impossible in a real-world production environment, such as passing null pointers to functions that are always protected by the rest of the application.

Overview of the agentic fuzzing workflow across writing, execution, and analysis stages.

This leads to a deluge of "false positive" crashes. In practice, up to 70% of initial crashes reported by automated systems can be "junk," wasting developer resources and undermining trust in automated security tooling. FalseCrashReducer addresses this by introducing two "agentic AI" strategies to proactively prevent and reactively filter these false positives.

Methodology: Proactive and Reactive Agents

The FalseCrashReducer framework integrates two specialized AI agents into the existing OSS-Fuzz-Gen pipeline to improve the fidelity of automated fuzzing.

The FalseCrashReducer architecture featuring constraint-based generation and context-based validation.

1. Proactive: The Function Analyzer Agent

Before generating a fuzz driver, the Function Analyzer Agent performs a deep dive into the codebase. Using automated search and grep tools, it identifies critical "constraints"—preconditions, necessary global state initializations, and buffer size requirements. These constraints are then provided as explicit instructions to the driver-generation agent, ensuring the resulting test code respects the target function's intended usage.

2. Reactive: The Crash Validator Agent

When a crash is detected, the Crash Validator Agent serves as a digital forensic investigator. It analyzes the stack trace and the project’s call graph to determine if the crashing state is actually reachable from the program's public entry points. If a crash can only be triggered through the artificial "back door" created by the fuzz driver, it is flagged as a false positive and filtered out.

To facilitate these interactions, the system utilizes a Shared Repository Pattern, allowing multiple agents to access and contribute to a centralized knowledge base of project-specific information.

Evaluation and Results

The authors evaluated FalseCrashReducer using a benchmark of 1,555 functions across 336 open-source projects within the OSS-Fuzz ecosystem. Powered by Gemini 2.5 Pro, the system demonstrated significant improvements in both driver quality and noise reduction.

  • Noise Filtering: The Crash Validator successfully filtered out 57.3% to 61.3% of reported crashes as infeasible, drastically reducing the manual triage burden.
  • Improved Driver Quality: The proactive constraint-based strategy ensured that 24.2% more drivers adhered to function requirements, leading to a 2–8% reduction in total (mostly false) crashes.
  • Economic Efficiency: The cost of generating a high-quality, validated fuzz driver is remarkably low, averaging less than $1.00 in API fees. This makes AI-driven security testing orders of magnitude more cost-effective than manual engineering.

Insights and Future Work

A key takeaway from this research is that frontier LLMs can effectively perform complex program analysis tasks that were previously the domain of specialized static analysis tools. However, the study also noted a "consistency gap," where the Crash Validator's conclusions were consistent only about 68% of the time, highlighting the ongoing challenge of LLM hallucinations.

Future work will focus on real-time constraint retrieval to provide even more context during driver generation and exploring AI-assisted bug fixing, where agents not only identify valid crashes but also propose the necessary code patches to resolve them.

Summary updated

BibTeX
@misc{amusuo2025falsecrashreducer,
  author = {Paschal C. Amusuo and Dongge Liu and Ricardo Andrés Calvo Méndez and Jonathan Metzman and Oliver Chang and James C. Davis},
  title = {{FalseCrashReducer: Mitigating False Positive Crashes in OSS-Fuzz-Gen Using Agentic AI}},
  year = {2025},
  eprint = {2510.02185},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2510.02185},
}
Figure