Retrofitting Applications with Provenance-Based Security Monitoring

Adam Bates, Kevin R. B. Butler, Alin Dobra, Bradley Reaves, Patrick T. Cable Ii, Thomas Moyer, Nabil ScheararXiv 2016 · arXiv

TL;DR

This paper introduces DAP, a system that retrofits existing web applications with provenance tracking capabilities without requiring code modifications. DAP enables real-time security monitoring, particularly for detecting and preventing SQL injection attacks, by providing detailed historical context of application workflows.

What Problem Does This Paper Address?

Modern software applications are complex and often built by composing many independent components. While data provenance (tracking the history of data and its transformations) is invaluable for security, forensics, and decision-making, deploying it in these complex, heterogeneous environments is challenging. Existing solutions often require extensive instrumentation of application code or rely on system-level provenance that suffers from a "semantic gap" – the system's view of events doesn't align with the application's logical operations. This makes it difficult to understand application-specific attacks like SQL injection. The authors aim to provide a holistic and affordable provenance solution for complex application workflows, specifically targeting web services.

Methods

DAP employs a multi-pronged approach to capture provenance non-invasively:

  • Execution Partitioning: A minimal modification to the web server (Apache 2) is made to identify and delineate "units of work" for each incoming request. This helps distinguish individual user sessions and requests, mitigating the "dependency explosion" problem where all events are assumed to depend on all prior events.
  • Database Capture Agent: This is a transparent TCP proxy that sits between the web application and the database. It intercepts SQL queries, parses them using a SQL grammar, and extracts provenance information about accessed (SQL_READ) and referenced (SQL_USED) data objects at the column and table level. It also records when data is generated (SQL_WASGENERATEDBY).
  • Provenance Recorder: This component aggregates provenance data from various sources (web server, database capture agent, and potentially system-level provenance) into a unified, in-memory graph. It uses a common namespace based on process IDs (PIDs) to link events across different layers.
  • Network Guard: This is another TCP proxy that monitors outbound network traffic. When a response is sent, it queries the Provenance Recorder for the ancestry of the data being transmitted. This allows for real-time policy enforcement, such as preventing sensitive data exfiltration.

Datasets

The paper does not explicitly mention training datasets. For evaluation, they used the Dell DVDStore Database Test Suite, an open-source simulation of an online e-commerce site. This benchmark was modified to issue SQL queries via curl requests to a toy PHP application on Apache 2, which then relayed them to MySQL. This setup allowed them to generate over 10,000 unique SQL queries to test DAP's performance and effectiveness.

Key Insights and Ideas

  • Minimal Intervention is Key: The core insight is that significant security benefits can be achieved with minimal changes to existing applications. DAP's success hinges on its ability to "retrofit" provenance without requiring developers to rewrite their code or databases.
  • Bridging the Semantic Gap: By combining web server execution partitioning with SQL query parsing, DAP effectively bridges the semantic gap between system-level events and application-level logic, providing more meaningful provenance for web workflows.
  • Column/Table Granularity is Sufficient: For many web security scenarios, tracking provenance at the column and table level (rather than individual database records) is sufficient and significantly reduces overhead. This granularity is effective for detecting attacks like SQL injection.
  • Real-time Defense is Achievable: The Network Guard component demonstrates that provenance can be used not just for post-mortem forensics but also for real-time threat detection and prevention, specifically for data exfiltration.
  • Garbage Collection for Scalability: The paper highlights the critical issue of provenance storage overhead. Their proposed garbage collection mechanism, which discards provenance for whitelisted benign requests, shows a significant reduction in storage requirements, making the system more scalable.

Key Results

  • Low Overhead: DAP imposes an average end-to-end delay of 5.1 ms (17.1%) on web requests during the DVDStore benchmark.
  • Microbenchmark Performance: Individual steps like SQL query parsing take an average of 0.11 ms. The primary delay comes from inter-process communication (IPC) for transmitting provenance events.
  • Storage Optimization: The garbage collection mechanism reduced provenance storage growth by 75% after 6 minutes of the DVDStore workload.
  • Network Guard Performance: The Network Guard can analyze provenance and process responses in an average of 1.23 ms, enabling real-time policy enforcement.
  • Case Studies: The paper demonstrates DAP's effectiveness in detecting and explaining:
  • SQL Injection: By analyzing provenance of data accessed and referenced in SQL queries.
  • ImageTragick Exploit: By combining DAP with system-layer provenance to track a remote shell invocation.
  • Reverse Shell Invocation: By identifying suspicious system commands executed by a web server worker.

Limitations and Future Work

  • Database Record Granularity: DAP does not track individual database record accesses. While this is a design choice for performance, it means it cannot pinpoint the exact records affected by a query.
  • NoSQL Databases: The current implementation focuses on relational databases and SQL. Extending DAP to support NoSQL databases is a direction for future work.
  • System Layer Provenance: While DAP can integrate with system-layer provenance (e.g., from a provenance-aware kernel), it assumes the integrity of this layer.
  • Garbage Collection Tuning: The garbage collection mechanism is a promising step, but further tuning and research are needed to address provenance storage for extremely high-traffic web services.
  • Policy Language: The policy language for the Network Guard is simple. More expressive policy languages could enhance its capabilities.

Summary updated

BibTeX
@misc{bates2016retrofitting,
  author = {Adam Bates and Kevin R. B. Butler and Alin Dobra and Bradley Reaves and Patrick T. Cable II and Thomas Moyer and Nabil Schear},
  title = {{Retrofitting Applications with Provenance-Based Security Monitoring}},
  year = {2016},
  eprint = {1609.00266},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1609.00266},
}

Loading related content...

Figure