TL;DR
This paper introduces ARROW, a novel system for detecting drive-by download attacks by identifying and generating signatures for "central servers" within malware distribution networks (MDNs). By analyzing URL patterns of these central servers, ARROW significantly boosts detection coverage of existing methods with a very low false positive rate.
What Problem Does This Paper Address?
Drive-by download attacks are a significant threat where users unknowingly download malware by simply visiting a compromised webpage. Attackers often use complex networks of servers (MDNs) to host malicious content and orchestrate these attacks. Existing detection methods primarily rely on analyzing the malicious content of exploit or malware distribution servers. However, these methods can be easily evaded by attackers who can detect the presence of these systems and serve benign content, or by the inherent instability of compromised servers. This leads to a high rate of false negatives. The paper aims to address this gap by developing a more robust detection mechanism that leverages the more stable infrastructure of MDNs, specifically focusing on identifying and utilizing "central servers" that are shared across a large percentage of attacks within the same MDN.
Methods
ARROW employs a multi-stage approach:
- Hostname-IP Mapping (HIC): To combat the diversity of hostnames and IP addresses used by attackers, ARROW groups hostnames that share a significant percentage of IP addresses into "Hostname-IP Clusters" (HICs). This allows for a more unified representation of servers.
- MDN Discovery and Central Server Identification:
- HTTP traces from high-interaction client honeypots are collected, recording visited URLs and whether a drive-by download was successful.
- HTTP traces are aggregated into potential MDNs based on either the malware's hash value or the HIC index of exploit URLs.
- MDNs with a minimum number of traces (THTTPTrace = 20) are retained to avoid noise.
- Within each MDN, "central servers" are identified as HICs that appear in a majority (Tcen = 0.9) of the traces.
- MDNs sharing central servers are merged to create larger, more robust MDNs.
- Regular Expression Signature Generation:
- For each identified central server, ARROW analyzes the URLs associated with it across all traces in its MDN.
- A signature tree is built based on common structural patterns (directory names, file names, extensions, parameters) within these URLs.
- "Critical nodes" in the tree are identified based on coverage thresholds.
- Branches leading to critical nodes are used to generate regular expression signatures that capture the invariant parts of the URLs, allowing for more flexible matching than exact string matching.
- Signatures are further pruned using a dataset of legitimate HTTP traces to reduce false positives.
Datasets
- HTTPTraces: A large dataset of 3.5 billion HTTP traces collected from a production cluster of high-interaction client honeypots.
- Smalicious: 1,345,890 traces identified as drive-by download attacks.
- Sunlabeled: The remaining traces, used as an unlabeled dataset.
- Sbenign1: 10,811,805 legitimate HTTP traces used for signature pruning.
- Sbenign2: 10,733,282 legitimate HTTP traces used for false positive evaluation.
Key Insights and Ideas
- Central Servers as Stable Anchors: The core insight is that while exploit servers might be transient or easily changed, the "central servers" that manage MDNs (e.g., for redirection or optimization) tend to be more stable and shared across many attacks. This stability makes them ideal targets for signature generation.
- URL Structure Over Content: By focusing on the structural patterns of URLs associated with central servers, ARROW bypasses the need to analyze potentially obfuscated or absent malicious content, making it more resilient to evasion.
- HIC for Server Normalization: The Hostname-IP Cluster (HIC) approach is crucial for abstracting away attacker attempts to obscure server identity through fast-flux or multiple hostnames, enabling more effective grouping and identification of central servers.
- Regular Expressions for Flexibility: Using regular expressions, rather than exact string matching, allows signatures to adapt to minor variations in URLs (like parameter values), significantly reducing false negatives.
Key Results
- Boosted Detection Coverage: ARROW signatures increased the detection coverage of existing methods by 96.0%.
- Low False Positive Rate: An extremely low false positive rate of 0.0019% was achieved on a large dataset of legitimate HTTP traces.
- Twitter Case Study: A signature for "twitter.com" (a legitimate site) was analyzed. While it matched a significant number of traces, a "ReflectRate" analysis indicated a very low potential false positive rate of 0.4% for this difficult case.
- MDN Identification: ARROW identified 6,937 MDNs from the malicious traces, with 97 MDNs (1.4%) employing central servers.
Limitations and Future Work
- Evasion Techniques: Attackers can still evade detection by generating different malware binaries (leading to different hash values) or by decentralizing MDNs to eliminate central servers or hide their structure.
- False Positives on Legitimate Sites: While the overall false positive rate is low, applying it to massive datasets can still result in a significant number of false positives, especially when legitimate sites are involved (as seen in the Twitter case study).
- Future Work:
- Refining the algorithm to aggregate polymorphic malware into MDNs using similarity of binaries or behavior characteristics.
- Developing better automated methods for assessing the purpose of central servers in the absence of a successful attack.
- Exploring strategies like white-listing or clustering based on page characteristics to further reduce false positives.
- Investigating more sophisticated MDN structures that might not rely on easily identifiable central servers.
Summary updated