Website Forensic Investigation to Identify Evidence and Impact of Compromise

Yuta Takata, Mitsuaki Akiyama, Takeshi Yagi, Takeshi Yada, Shigeki GotoSecureComm 2016 · DOI

TL;DR

This paper introduces a novel "redirection call graph" (RCG) approach to precisely pinpoint compromised web content and understand the impact of website compromises, particularly for drive-by download attacks. By analyzing websites across multiple client environments, their system provides crucial, fine-grained information that significantly aids incident responders and webmasters in cleaning up compromised sites more efficiently.

What Problem Does This Paper Address?

Attackers frequently compromise popular websites to host malicious content, often employing "drive-by download" schemes that lure unsuspecting users to malware-infected sites. While security organizations (like CSIRTs) and webmasters work to combat this, current methods for identifying compromised websites have significant limitations. Abuse reports often contain only URLs, which are insufficient for webmasters to locate the exact compromised content. Furthermore, malicious websites can dynamically alter their behavior based on the user's environment (e.g., browser version, plugins), making analysis difficult and leading to "cloaking" techniques that evade detection. This lack of detailed information hinders prompt incident response and effective cleanup.

Methods

The core contribution of this paper is the Redirection Call Graph (RCG). Unlike traditional redirection graphs that primarily rely on HTTP headers (like Referer) or simple JavaScript analysis, the RCG combines information from both HTTP transactions and JavaScript execution traces.

Key aspects of their method include:

  • Monitoring Behavior: The system monitors HTTP transactions (capturing 3XX redirects) and HTML parsing (identifying redirecting tags like <iframe> and <script>). Crucially, it also hooks JavaScript API calls (e.g., eval, setTimeout, document.write) to trace code execution.
  • Constructing RCG: This phase builds a directed graph where nodes represent URLs and JavaScript execution events. Edges represent redirection methods or JavaScript execution flows.
  • Bridging Semantic Gaps: A key innovation is the "semantic gap edge." This allows the RCG to correctly link JavaScript code that dynamically generates HTML tags (e.g., document.write('<iframe></iframe>')) to the actual redirection that occurs via that tag. This is a significant improvement over conventional methods that might miss this connection.
  • Identifying Malicious Nodes: The RCG is then analyzed using a blacklist of known malicious URLs to identify malicious paths.
  • Extracting Compromised Content: By traversing malicious paths backward, the system identifies the precise web content (HTML, JavaScript) responsible for the redirection.
  • Identifying Targeted Client Environments: The system uses a multi-client environment to simulate various user configurations (browser, plugins) to detect client-dependent redirections and identify which environments are targeted by malicious sites. This is achieved by minimizing environment profiles based on known vulnerability information.

Datasets

The paper evaluates its system using a dataset of approximately 2,000 compromised websites that were preliminarily detected over a four-year period (2011-2015). This dataset was derived from HTTP communication data captured by a high-interaction honeyclient.

Key Insights and Ideas

  • The "Semantic Gap" is a Major Hurdle: The paper highlights how legitimate web browser behavior (e.g., JavaScript dynamically creating HTML) creates a "semantic gap" that conventional redirection graph methods struggle to bridge, leading to incomplete or incorrect analyses.
  • RCG Solves the Semantic Gap: The RCG's ability to link JavaScript execution to HTML tag generation is crucial for accurately mapping complex redirection chains.
  • Client-Dependent Redirection is Prevalent: A significant portion of compromised websites exhibit client-dependent behavior, meaning they redirect users differently based on their browser, plugins, or OS. This underscores the need for multi-client analysis.
  • Fine-Grained Information is Key: Simply providing a URL is insufficient. The precise location of compromised content, the chain of redirections, and the targeted client environments are vital for effective incident response.

Key Results

  • The RCG approach successfully identified malicious paths in 71.9% of the analyzed compromised websites.
  • The system successfully identified compromised web content and malicious URL relations, leading to a reduction in the amount of web content and number of URLs to be analyzed by 0.8% and 15.0%, respectively, for incident responders.
  • The system could identify client-dependent redirections and targeted client environments in 30.4% of websites.
  • Using target information, the system could also identify vulnerabilities exploited by malicious websites.
  • The RCG approach identified 1,068 and 367 more nodes (URLs) in malicious paths compared to conventional referer-based and heuristic-based methods, respectively.
  • The semantic gap edge was present in 33% of analyzed redirection paths.

Limitations and Future Work

  • Browser Emulator Limitations: The system, like other browser emulators, cannot execute exploit code, meaning it cannot fully trace malware distribution URLs or construct complete redirection graphs for all attacks. It also faces challenges with sophisticated browser fingerprinting and emulator evasion techniques.
  • Dynamic Content: Content dynamically injected from databases or .htaccess files cannot be accurately identified.
  • User Study Needed: The authors acknowledge the lack of a user study to quantify how their identified information directly impacts response rates and cleanup times for webmasters.
  • Future Work: The authors suggest combining their system with other techniques like honeyclients and machine learning to maximize malicious content disclosure and detection. They also plan to conduct user studies to evaluate the practical impact on incident responders and webmasters.

Summary updated

BibTeX
@inproceedings{takata2016website,
  author = {Yuta Takata and Mitsuaki Akiyama and Takeshi Yagi and Takeshi Yada and Shigeki Goto},
  title = {{Website Forensic Investigation to Identify Evidence and Impact of Compromise}},
  booktitle = {SecureComm},
  year = {2016},
  doi = {10.1007/978-3-319-59608-2\_25},
}

Loading related content...

Figure