TL;DR
This paper argues that the common practice of measuring abuse (like phishing) by simply counting incidents is misleading. Instead, they propose a model that disentangles structural factors of hosting providers from their actual security efforts, finding that structural properties explain the vast majority of abuse variations. This shift in perspective is crucial for developing effective cybersecurity policies and interventions.
What Problem Does This Paper Address?
Cybersecurity research and policy often rely on "abuse data" – indicators of malicious activity like spam, malware, or phishing – to assess the security posture of online service providers, particularly hosting providers. A common assumption is that high concentrations of abuse indicate poor security practices or even deliberate negligence by the provider, leading to labels like "rotten apples." However, this approach faces a significant challenge: readily available abuse data is often noisy and misaligned with actual information needs. Factors like the sheer size of a provider's infrastructure (number of hosted websites, IP addresses) can inflate abuse counts, making it difficult to discern whether a provider is genuinely insecure or simply larger. This paper tackles the problem of accurately attributing abuse to either the provider's security efforts or their inherent structural characteristics, aiming to move beyond simplistic interpretations of raw abuse metrics.
Methods
The authors develop a novel analytical approach and a statistical model to decompose the sources of variance in abuse data. They move beyond simple regression models by:
- Scalable Approach: Utilizing passive DNS and WHOIS data to study abuse across the entire population of hosting providers, rather than just technical entities like Autonomous Systems.
- Decomposition of Variance: Carefully breaking down the sources of variance in abuse data into explanatory factors (structural properties, security efforts, attacker behavior) and measurement errors (misattribution, observation biases).
- "Statistical Twins" Sampling: To overcome the cost and scalability issues of collecting detailed data for all providers, they employ a "statistical twins" sampling method. This involves selecting pairs of providers that are similar in observable structural properties, allowing for a more focused analysis of differences.
- Poisson Generalized Linear Models (GLMs): They use Poisson GLMs with a log-link function to model the count of abuse incidents (specifically phishing domains) per provider. This allows them to estimate the impact of various explanatory variables while accounting for the nature of count data.
- Controlling for Heterogeneity: The models incorporate fixed effects for both statistical twins and countries to control for unobserved heterogeneity and country-level differences, respectively.
Datasets
- APWG (Anti-Phishing Working Group) Data: Used as the primary dataset for phishing domain counts. This dataset contains global phishing domain information, blacklisting times, and associated IP addresses.
- Cyscon GmbH Data: Used for cross-validation of results. This dataset also includes URLs, IP addresses, and blacklisting times for phishing domains.
- Passive DNS Data (DNSDB): Used to map IP addresses and domains to hosting providers.
- WHOIS Data: Used for identifying hosting providers and collecting information on their domain registration dates.
- MaxMind API: Used for identifying IP address locations.
- Alexa Top-Million Websites: Used to derive a "popularity index" for hosted domains.
- Internet Archive (Web-Archive): Used to verify time-in-business data.
Key Insights and Ideas
- Structural Factors Dominate: The most striking insight is that structural properties of hosting providers (like size and business model) explain a disproportionately large amount of the variation in abuse counts (84% in their initial model). This challenges the common narrative that "rotten apples" are the primary drivers of abuse.
- "Bad Harvest" Over "Rotten Apples": The findings suggest that the overall "harvest" (the industry's structural landscape) is a much stronger determinant of abuse levels than individual "rotten apples" (insecure providers).
- Importance of Contextualized Metrics: Absolute abuse counts are misleading. The paper advocates for "comparative abuse metrics" that control for structural differences, providing a more accurate way to evaluate provider performance and inform policy.
- Unstudied Factors Matter: Beyond size, factors like pricing (cheaper hosting correlates with more abuse), time in business, and the prevalence of popular platforms like WordPress significantly influence abuse rates, even after controlling for structural factors.
- Policy Implications: The findings suggest that policy interventions should focus on improving the overall "harvest" (e.g., requiring higher security standards for providers hosting popular websites) rather than solely targeting individual "bad actors."
Key Results
- 84% Variance Explained by Structural Properties: In their initial model using data from 45,358 hosting providers, four structural properties (number of assigned IPs, IPs hosting domains, hosted domains, and percentage of domains on shared IPs) explained 84% of the variation in phishing counts.
- Additional Factors Explain More: When incorporating additional factors like pricing, time in business, popularity index, and WordPress use on a sample of "statistical twins," an additional 77% of the remaining variation in abuse was explained.
- Negative Correlation with Price: Hosting providers with higher prices showed significantly less abuse, suggesting that cost is a proxy for resources that can be allocated to security.
- Positive Correlation with Popularity and WordPress: Providers hosting more popular websites and a higher proportion of WordPress sites experienced more abuse, likely due to these being common targets for compromise.
- Negative Correlation with Assigned IPs (when controlling for size): Providers with a larger number of assigned IP addresses (but not necessarily using them for web hosting) showed less abuse, suggesting they might be more like broadband providers with a smaller web hosting footprint.
Limitations and Future Work
- Limitations:
- The "statistical twins" sampling method might miss extreme outliers.
- The assumption of linear bias introduced by the sampling strategy might not always hold.
- The study is specific to phishing; other types of abuse might have different drivers.
- While structural factors are dominant, the presence of unobserved factors (like attacker behavior) and measurement errors still contribute to unexplained variance.
- Future Work:
- Apply the developed approach to other forms of hosting abuse (e.g., drive-by-download sites).
- Investigate if "rotten apples" are more prevalent for specific, more idiosyncratic types of abuse (e.g., long-living botnet C&C servers).
- Explore more direct measurement of provider security efforts through surveys or experiments.
- Further refine attribution methods to better distinguish between structural factors and security efforts.
Summary updated