CrossPloit: A Benchmark for Cross-Version PoC Migration in Python Web Applications

Hung Nguyen-Huu, Nguyen Anh Tuan, Ting Zhang, Yue Liu, Chengran Yang, Lwin Khin Shar, Eng Lieh Ouh, David LoarXiv 2026 · arXiv

TL;DR

Vulnerability databases like NVD often have "blind spots" where older, unsupported software versions are incorrectly assumed to be safe. CrossPloit is a new benchmark of 127 migration instances that challenges LLM agents to prove these versions are exploitable by adapting existing Proof-of-Concept (PoC) exploits. The study reveals that while top-tier models like Claude Opus-4.6 can resolve up to 58.3% of these cases, complex architectural shifts in attack vectors remain a significant hurdle for current automation.

What Problem Does This Paper Address?

Modern security relies on "affected-version" lists. If you run a version of a library that isn't listed in a CVE entry, your scanners, patching pipelines, and incident responders will likely treat it as safe. However, these lists are frequently incomplete, especially for older branches that have reached "end-of-life" but are still widely deployed.

Proving a version is vulnerable requires executable evidence: a working PoC. But you can't just "replay" an exploit from a newer version. APIs get renamed, request paths change, and the environment (like Python or database versions) might be fundamentally different. This task, called cross-version PoC migration, is a complex puzzle involving source-code analysis, environment repair, and execution-based verification.

Methods

The authors developed a rigorous pipeline to build and test the CrossPloit benchmark:

  1. PoC Generation: They reconstructed 97 reference PoCs for Python web CVEs from scratch, ensuring each had a deterministic, vulnerability-specific "oracle" (a way to programmatically check if the exploit worked).
  2. Manual Migration: Human experts attempted to migrate these PoCs to five prior versions not listed as affected. This resulted in 127 validated migration instances across 45 CVEs.
  3. Agent Evaluation: They tested 18 different LLM agent configurations using the OpenHands and SWE-agent harnesses, paired with nine state-of-the-art models (e.g., GPT-5.3, Claude Opus-4.6).
  4. Baseline Comparison: They adapted VulScope, a trace-guided migration tool originally for C/C++, to the Python web setting (VulScope-Web) to see how traditional input-mutation techniques compare to LLMs.
Figure 2: Overview of CrossPloit construction pipeline

Datasets

The primary contribution is the CrossPloit benchmark:

  • Scope: 127 migration instances across 45 CVEs in the Python web ecosystem (Django, Flask, FastAPI, Web2py, etc.).
  • Components: Each instance includes a version-pinned Docker container, a reference PoC, and a vulnerability-specific oracle.
  • Diversity: The benchmark covers various vulnerability types, dominated by SQL injection (CWE-89), XSS (CWE-79), and Path Traversal (CWE-22).
Figure 3: Vulnerability profile of the 45 benchmark CVEs.

Key Insights and Ideas

  • The Massive Blind Spot: Benchmark construction confirmed 149 vulnerable prior versions that were absent from all four major vulnerability databases (NVD, OSV, GHSA, Snyk).
  • Replay is Insufficient: Only about 15% of these versions could be exploited by simply re-running the original PoC. The rest required active migration (code changes).
  • The "Attack Vector Shift" Wall: The hardest migrations are those where the original entry point is gone, but the bug is still reachable through a different path. No agent configuration could solve more than 37.9% of these cases.
  • Context Matters: An ablation study showed that withholding the reference PoC and environment info caused the resolve rate to plummet from 50.4% to 15.0%. LLMs aren't just "finding" bugs; they are heavily reliant on the provided templates to bridge version gaps.
  • Success Drops with Distance: As the version distance from the disclosed version increases, the likelihood of a successful migration (and the presence of the vulnerability itself) decreases significantly.
Outcome distribution by version distance

Key Results

  • Top Performer: The best configuration was OpenHands with Claude Opus-4.6, which resolved 58.3% (74/127) of the migration instances.
  • Baseline Failure: The adapted trace-guided baseline (VulScope-Web) only resolved 8.7% of instances, primarily because it can only mutate inputs and cannot fix broken APIs or environments.
  • Failure Analysis: Higher-performing models were better at identifying the correct API/entry point but still often failed at the "last mile"—completing the execution and verifying it with the oracle.
  • Human Effort: Constructing the benchmark was labor-intensive, requiring approximately 242 person-hours for PoC generation and 291 hours for migration attempts.

Limitations and Future Work

  • Ecosystem Scope: The study is currently limited to the Python web ecosystem. The authors note that results might differ in other languages or application types.
  • Evaluation Depth: Each tool was run only once per instance; repeated runs are needed to assess the stability of these resolve rates.
  • Metadata Auditing: While the benchmark focuses on confirmed vulnerable versions, a real-world auditing tool would also need to be evaluated on its ability to correctly identify unaffected versions (minimizing false positives).
  • Future Directions: The authors suggest extending the benchmark to other ecosystems and improving agent support for historical dependency resolution, which remains a persistent cause of failure.

Summary updated

Abstract

Affected-version ranges in vulnerability databases decide which deployments scanners, patching pipelines, and incident responders treat as vulnerable, yet these ranges are often incomplete, so vulnerable releases are silently treated as safe. Auditing them requires executable evidence: given a proof-of-concept (PoC) exploit for a listed version, show that the same vulnerability can be triggered on an earlier, unlisted version. This cross-version PoC migration combines source-code analysis, environment repair, and execution-based verification. We introduce CrossPloit, a benchmark of 127 migration instances across 45 CVEs in the Python web ecosystem. Each instance includes a version-pinned container, a reference PoC, and a vulnerability-specific oracle; two validators check that the observed effect arises from the disclosed vulnerability. Benchmark construction confirms 149 vulnerable prior versions absent from the affected-version lists of four databases, 127 of which require migration. Across 18 agent configurations (nine models and two harnesses), the best configuration resolves 58.3% of instances, compared with 8.7% for an adapted trace-guided baseline. Attack-vector shifts are particularly difficult: no configuration resolves more than 37.9% of these instances. Higher resolve rates are associated with a smaller share of missed-API failures and a larger share of incomplete executions among the remaining failures. In one configuration, withholding the reference PoC and its environment specification reduces resolve rate from 50.4% to 15.0%, highlighting the value of the supplied reference package. CrossPloit provides an execution-based measurement of under-reported exposure and of how far its enumeration can be automated.
BibTeX
@misc{nguyenhuu2026crossploit,
  author = {Hung Nguyen-Huu and Nguyen Anh Tuan and Ting Zhang and Yue Liu and Chengran Yang and Lwin Khin Shar and Eng Lieh Ouh and David Lo},
  title = {{CrossPloit: A Benchmark for Cross-Version PoC Migration in Python Web Applications}},
  year = {2026},
  eprint = {2611.16588},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.16588},
}
Figure