SCWAD: Automated Pentesting of Web Applications

Han Yufei, Natan Talon, Valérie Viet Triem Tong, Gilles Guette, Yufei Han, Youssef LaarouchiSecrypt 2024 · DOI

TL;DR

This paper introduces SCWAD, an automated framework for web application pentesting that models the process as a sequential decision-making problem. By integrating a knowledge base, an intelligent agent, and a verifier, SCWAD aims to provide more comprehensive vulnerability detection than traditional automated tools, even for large, complex applications.

What Problem Does This Paper Address?

Web applications are ubiquitous and critical, but they are also frequent targets for cyberattacks. While automated tools like scanners and fuzzers offer speed and cost-effectiveness for security assessments, they often struggle to detect complex vulnerabilities like broken access control and can produce many false positives. Manual penetration testing, on the other hand, is thorough but time-consuming and requires significant expertise. This creates a gap where a more automated, yet comprehensive, approach to web application security assessment is needed. The paper addresses the challenge of effectively organizing and executing web application pentests, which are inherently complex due to the diverse nature of applications and the cost of manual testing.

Methods

SCWAD is built around three core components that interact in a continuous loop:

  • Knowledge Base: This component stores structured information about the web application, including its IP address/domain, user roles, credentials, cookies, and visited pages with their details (path, HTTP methods, headers, parameters, and HTML content). This knowledge base is crucial for understanding the application's state.
  • Agent: This automated component is responsible for exploring the application and attempting simulated attacks. It compiles a list of potential actions (e.g., accessing a web page, submitting a form, setting a cookie) based on the current knowledge base, selects an action based on a chosen strategy (random, explore first, exploit first, or vulnerability tracking), and executes it.
  • Verifier: This component checks for new vulnerabilities after each action executed by the agent. Vulnerabilities are represented as Boolean formulas in first-order logic, and the verifier checks if these formulas hold true based on the updated knowledge base.

A key innovation is framing pentesting as a sequential decision-making problem, where the agent intelligently chooses actions to explore the application and discover vulnerabilities. The "vulnerability tracking" strategy is particularly novel, mimicking human pentester behavior by learning from past actions and sequences to guide future exploration.

Datasets

The paper evaluates SCWAD on three web applications:

  • Damn Vulnerable Web Application (DVWA): A deliberately vulnerable application designed for security professionals to test their skills.
  • WackoPicko: A simulated commercial web application designed to mimic real-world scenarios.
  • UVVU: A custom-built, intentionally poorly designed web application created by the authors to embed vulnerabilities and serve as a testbed for SCWAD.

These applications were chosen because they implement various vulnerability types, including Technical Information Disclosure (TID), Broken Access Control (BAC), and Reflected Cross-Site Scripting (XSS). The authors manually verified the exploitability of these vulnerabilities to establish a ground truth for their experiments.

Key Insights and Ideas

  • Pentesting as Sequential Decision-Making: The core idea of modeling pentesting as a sequential decision-making process is a significant conceptual shift, allowing for more intelligent and adaptive exploration.
  • Knowledge Base as a Central Hub: The structured knowledge base is critical for SCWAD's ability to understand the application's state, user interactions, and potential attack paths.
  • Vulnerability Tracking Strategy: This strategy, which mimics human pentester behavior by learning from past actions and sequences, significantly outperforms other strategies in discovering vulnerabilities, especially complex ones.
  • Automated BAC Detection: SCWAD demonstrates a unique capability to detect Broken Access Control vulnerabilities, a feat that traditional automated scanners struggle with.
  • Reduced False Positives: SCWAD aims to produce fewer false positives compared to traditional scanners by focusing on genuine vulnerabilities.

Key Results

SCWAD was compared against three state-of-the-art tools: BurpSuite Pro, OWASP ZAP, and Wapiti.

  • XSS Detection: SCWAD performed comparably to BurpSuite Pro, achieving high True Positive Rates (TPR) and low False Negative Rates (FNR) across the tested applications. It also demonstrated a lower False Positive Rate (FPR) in some cases, indicating better accuracy.
  • Broken Access Control (BAC) Detection: SCWAD was the only tool that successfully detected BAC vulnerabilities, achieving a 100% TPR. This highlights its advantage in handling multi-user scenarios.
  • Technical Information Disclosure (TID) Detection: SCWAD outperformed other tools in detecting TID vulnerabilities, particularly by triggering errors that revealed stack traces, which other tools missed.
  • Performance of Strategies: The "Vulnerability Tracking" strategy achieved a 100% success rate in uncovering all tested vulnerabilities on the UVVU application, outperforming random, explore-first, and exploit-first strategies.

Limitations and Future Work

  • Limitations:
    • SCWAD's current implementation may not fully handle vulnerabilities triggered solely by JavaScript events (as seen in the BurpSuite comparison).
  • The "Oracle" component, which generates XSS payloads, might require further enrichment to find all vulnerabilities in certain scenarios.
  • Processing large amounts of page data can be computationally intensive, suggesting a need for page clustering and better data "understanding."
    • Ethical considerations were addressed by not uploading payloads to real-world applications and by sharing findings responsibly.
  • Future Work:
    • Integrating reinforcement learning-based agents to enhance adaptability and efficiency in vulnerability exploration.
  • Developing more sophisticated methods for understanding and processing web application data to improve performance.
  • Further enriching the Oracle component with more advanced payload generation techniques.

Summary updated

BibTeX
@inproceedings{yufei2024scwad,
  author = {Han Yufei and Natan Talon and Valérie Viet Triem Tong and Gilles Guette and Yufei Han and Youssef Laarouchi},
  title = {{SCWAD: Automated Pentesting of Web Applications}},
  booktitle = {Secrypt},
  year = {2024},
  doi = {10.5220/0012721000003767},
}

Loading related content...

Figure