The Age of DDoScovery: An Empirical Comparison of Industry and Academic DDoS Assessments

Raphael Hiesgen, Marcin Nawrocki, Marinho P. Barcellos, Daniel Kopp, Oliver Hohlfeld, Echo Chan, Roland Dobbins, Christian Doerr, Christian Rossow, Daniel R. Thomas, Mattijs Jonker, Ricky K. P. Mok, Xiapu Luo, John Kristoff, Thomas C. Schmidt, Matthias Wählisch, K. C. ClaffyIMC 2024 · arXiv

TL;DR

This paper tackles the challenge of understanding the DDoS landscape by empirically comparing industry and academic data sources. It reveals significant discrepancies and limitations in visibility across different vantage points, highlighting the critical need for greater data sharing and collaboration to build a more accurate and unified view of DDoS threats.

What Problem Does This Paper Address?

The world of Distributed Denial-of-Service (DDoS) attacks is vast and complex, with a constant evolution of attack strategies and mitigation techniques. While both academic researchers and industry players produce reports and data on DDoS trends, there's a significant lack of consensus and often conflicting information. This makes it difficult to get a clear, unified picture of the true scale, nature, and evolution of DDoS attacks. The authors aim to bridge this gap by directly comparing data from various academic and industry sources, identifying the reasons for discrepancies, and proposing a path towards better understanding and collaboration.

Methods

The authors employ a multi-pronged approach:

  • Industry Report Analysis: They systematically reviewed 24 industry reports from 2022 to extract trends and identify inconsistencies in how commercial stakeholders report on DDoS.
  • Longitudinal Data Comparison: They analyzed ten diverse datasets spanning four years (2019-2023) from various sources, including network telescopes, flow monitoring (from DDoS mitigation providers), and honeypots. This allowed them to compare trends and identify discrepancies based on data sources, vantage points, methods, and parameters.
  • Novel Data Sharing Approach: A key innovation is their method of sharing an aggregated list of DDoS targets with industry players. These players then use their proprietary data to identify which targets are visible to them but not captured by the academic data sources, thus revealing gaps in academic visibility.
  • Statistical Analysis: They used techniques like Spearman correlation to quantify the agreement between different observatories' time series data.

Datasets

The study utilizes ten datasets from seven observatories, covering a period from 2019 to mid-2023:

  • Network Telescopes: UCSD NT and ORION NT (measuring randomly-spoofed DoS attacks).
  • Flow Data (DDoS Mitigation Providers): Netscout Atlas and Akamai Prolexic (measuring direct-path and reflection-amplification attacks). IXP Blackholing data was also used.
  • Honeypots: Hopscotch HP and AmpPot HP (measuring reflection-amplification attacks).

Key Insights and Ideas

  • Visibility Gaps are Pervasive: No single observatory, whether academic or industry, has a complete view of the DDoS landscape. Different vantage points capture different facets of attacks, leading to significant discrepancies in reported trends.
  • Industry vs. Academic Reporting: Industry reports often lack methodological transparency and can be influenced by marketing concerns, making direct comparison with academic findings challenging.
  • The Value of Federated Inference: Combining data from multiple sources, even with their limitations, is crucial for a more robust understanding. The authors' approach of sharing target lists with industry partners is a promising step towards this.
  • DDoS Trends are Not Monolithic: While some general trends emerge (e.g., an increase in direct-path attacks over time), the specifics vary greatly depending on the observatory and attack type.
  • Source Address Validation (SAV) Impact: The study provides evidence that industry efforts to promote SAV may have contributed to a reported drop in spoofed reflection-amplification attacks.

Key Results

  • Direct-Path Attacks: Four out of five observatories showed an upward trend in direct-path attacks over the 4.5-year period. However, the specific timing and magnitude of peaks varied significantly.
  • Reflection-Amplification Attacks: A general trend of increased attacks in 2020, followed by a decrease in 2021, was observed across most observatories. However, trends diverged in 2022 and 2023.
  • Target Overlap: A small fraction of targets (0.55%) was observed by all four academic observatories. Industry data (Netscout) showed a larger overlap (20%) with the set of targets seen by all academic observatories, suggesting that larger, multi-vector attacks are more likely to be seen across different vantage points.
  • Correlation Analysis: Observatories measuring the same attack type generally showed higher correlations, but significant variations existed, even within the same category. Akamai's direct-path data showed an unusual correlation with reflection-amplification observatories.

Limitations and Future Work

  • Data Sharing Obstacles: The authors acknowledge that obtaining and comparing data across different institutions is challenging due to privacy concerns, proprietary interests, and a lack of standardized data governance models.
  • Methodological Differences: Variations in attack detection algorithms, parameters, and vantage points inherently lead to different inferences, making direct comparisons difficult.
  • Limited Industry Transparency: The lack of detailed methodological information in industry reports hinders a deeper understanding of their findings.
  • Future Work: The authors advocate for greater transparency and data sharing, proposing recommendations for public policy and industry self-regulation. They also aim to maintain a living compilation of DDoS literature and encourage others with DDoS data to contribute to a more holistic understanding of the ecosystem.

Summary updated

BibTeX
@inproceedings{hiesgen2024age,
  author = {Raphael Hiesgen and Marcin Nawrocki and Marinho P. Barcellos and Daniel Kopp and Oliver Hohlfeld and Echo Chan and Roland Dobbins and Christian Doerr and Christian Rossow and Daniel R. Thomas and Mattijs Jonker and Ricky K. P. Mok and Xiapu Luo and John Kristoff and Thomas C. Schmidt and Matthias Wählisch and K. C. Claffy},
  title = {{The Age of DDoScovery: An Empirical Comparison of Industry and Academic DDoS Assessments}},
  booktitle = {IMC},
  year = {2024},
}
Figure