Locate-Then-Detect: Real-time Web Attack Detection via Attention-based Deep Neural Networks

Tianlong Liu, Yu Qi, Liang Shi, Jianan YanIJCAI 2019 · DOI

TL;DR

This paper introduces "Locate-Then-Detect" (LTD), a novel deep learning system for real-time detection of web attacks like SQL Injection and Cross-Site Scripting. LTD efficiently pinpoints suspicious attack payloads within long web requests, significantly improving accuracy and reducing computational costs compared to existing methods.

What Problem Does This Paper Address?

Web applications are constantly under threat from attacks like SQL Injection (SQLi) and Cross-Site Scripting (XSS). These attacks often hide short, malicious "payloads" within much longer, legitimate web requests (URLs, POST data). Traditional security systems struggle with this for two main reasons:

  1. Payload Size vs. Request Size: The tiny size of attack payloads makes it hard for models to learn meaningful patterns when they have to process the entire, often thousands-of-characters-long, web request.
  2. Real-time Performance: Detecting these attacks needs to happen instantaneously to protect users and applications, which is a challenge for computationally intensive machine learning models.

Existing solutions like traditional Web Application Firewalls (WAFs) rely on manually crafted rules, which are difficult to maintain and often fail to detect novel, "zero-day" attacks. While data-driven approaches like Libinjection and anomaly detection systems (e.g., HMM-Web) show promise, they still face limitations in accuracy, speed, or the ability to pinpoint the exact attack location.

Methods

The core innovation of LTD is its "Locate-Then-Detect" (LTD) framework, which consists of two main deep neural network modules:

  1. Payload Locating Network (PLN): This module acts like a smart highlighter. It takes a web request as input and efficiently proposes the most suspicious regions (potential attack payloads) within the long request. This is inspired by region proposal networks used in computer vision. The PLN uses character-level embeddings, a variant of the Xception model for feature extraction, and a sliding window approach with "anchors" to predict the start and end positions of suspicious segments.
  2. Payload Classification Network (PCN): Once the PLN identifies suspicious regions, the PCN takes these smaller, targeted segments and classifies them as either malicious (SQLi, XSS) or benign. The PCN uses a multi-layer convolutional neural network with diverse filters and max-overtime pooling to capture rich features from the proposed regions.

By first locating potential attack fragments, the PCN can focus its analysis on these critical areas, ignoring the vast majority of irrelevant "background" data. This two-stage approach significantly reduces computational load and improves the model's ability to detect subtle attack patterns.

Datasets

The paper evaluates LTD on two types of datasets:

  • CSCI 2010 Benchmark Dataset: This dataset contains generated traffic for an e-commerce web application, with manually reviewed samples labeled for SQLi and XSS attacks. It includes over 25,000 anomalous requests and 36,000 benign requests.
  • Real-World Web Traffic Dataset (RealDataset): This is a newly constructed dataset comprising 3 million web traffic logs from eight web servers in a busy cloud environment, with 38,600 attack samples (SQLi or XSS) labeled by security experts.

Key Insights and Ideas

  • The Power of "Locating": The most significant insight is that explicitly locating suspicious regions before classification dramatically improves both efficiency and accuracy. By filtering out irrelevant data, the system can focus on what matters.
  • Bridging the Gap Between Anomaly Detection and Signature-Based Methods: LTD combines the strengths of both. The PLN acts somewhat like an anomaly detector by highlighting unusual patterns, while the PCN then uses learned "signatures" (features) to classify these highlighted regions.
  • Interpretability through Location: Unlike traditional black-box deep learning models, LTD's ability to pinpoint the exact location of an attack payload makes it more interpretable, aiding security analysts in understanding and debugging detections.
  • Human-in-the-Loop for Zero-Day Detection: The paper proposes a human-in-the-loop fine-tuning mechanism to continuously improve the models, potentially helping to detect novel or "zero-day" attacks that haven't been seen before.

Key Results

  • Efficiency: LTD achieved an average detection time of 5ms on GPUs, making it well-suited for real-time applications. It reduced computational costs by 82.6% compared to processing full requests.
  • Accuracy:
    • On the CSCI benchmark, LTD outperformed an HMM-based method, Libinjection, and a commercial WAF, achieving high precision and recall (F1-score of 0.991).
  • On the real-world dataset, LTD achieved the highest precision and an F1-score of 0.999, with a very low false positive rate (FPR) of 0.0000%.
  • Comparison to Char-level CNN: LTD showed significantly lower FPR and False Negative Rate (FNR) compared to a char-level CNN approach that processes raw requests, demonstrating the benefit of the payload locating step.

Limitations and Future Work

  • Limited Attack Types: Currently, LTD is designed to detect only SQLi and XSS attacks.
  • Future Work: The authors plan to extend LTD to detect a wider range of web hacks, including File Inclusion and Code Execution attacks. They also emphasize the ongoing work on human-in-the-loop fine-tuning for better zero-day detection capabilities.

Summary updated

BibTeX
@inproceedings{liu2019locate,
  author = {Tianlong Liu and Yu Qi and Liang Shi and Jianan Yan},
  title = {{Locate-Then-Detect: Real-time Web Attack Detection via Attention-based Deep Neural Networks}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2019},
  doi = {10.24963/ijcai.2019/656},
}
Figure