DCNetBench: Scaleable Data Center Network Benchmarking

Ke Liu, Wanling Gao, Chunjie Luo, Cheng Huang, Chunxin Lan, Zhenxing Zhang, Lei Wang, Xiwen He, Nan Li, Jianfeng ZhanarXiv 2023 · DOI · arXiv

TL;DR

Benchmarking data center networks is notoriously difficult due to the complexity of real-world traffic and the proprietary nature of network data. This paper introduces DCNetBench, a novel framework that generates realistic, scalable network traffic traces by emulating data center environments and running actual applications, offering a more representative and adaptable solution for network evaluation.

What Problem Does This Paper Address?

Data center networks are the backbone of modern digital infrastructure, but accurately benchmarking them is a significant challenge. Existing methods fall short: RFC specifications and mathematical models fail to capture the nuances of real-world traffic, while industry-provided traces are often outdated, limited in scope, or not publicly available. Furthermore, the sheer scale and complexity of data centers make direct testing impractical. The core problem is the lack of a benchmarking methodology that is both realistic and adaptable to diverse applications, network configurations, and evolving technologies.

Methods

DCNetBench tackles this by proposing a scaleable, application-driven benchmarking methodology and framework. The core idea is to:

  • Emulate Realistic Environments: They use MaxiNet, a distributed network emulator, to create virtual data center environments with configurable network topologies, protocols, and hardware. This allows for large-scale simulations on limited physical resources.
  • Run Real-World Applications: Instead of relying on abstract models, DCNetBench executes actual applications (from benchmarks like BigDataBench and AIBench) within the emulated environment.
  • Monitor and Analyze Traffic: Runtime network traffic is captured using TCPDUMP. This raw data is then analyzed to extract key information like timestamps, packet sizes, and port information.
  • Classify and Model Patterns: The analyzed traffic is classified based on network patterns (e.g., stable, burst, increase) and characteristics. This forms the basis for generating representative traffic traces.
  • Generate Micro and Component-Level Traces: DCNetBench creates two types of traces: "micro traces" representing single traffic patterns (like a burst) and "component-level traces" representing a combination of patterns from a full application execution.
  • Automatic Benchmarking Framework: A four-module framework (environment deployer, trace generator, trace evaluator, metric reporter) automates the entire process, from emulation to performance evaluation.

Datasets

The paper introduces DCNetBench as a new benchmark. It leverages applications from:

  • BigDataBench [21]: Including Grep, WordCount, Random Sampling, PageRank, and Matrix Multiplication.
  • AIBench [22, 23]: Including an Image Classification application.

These applications are run across three network topologies: three-tier tree, fat-tree, and spine-leaf. The generated traces are categorized into twelve micro traces and four component-level traces.

Key Insights and Ideas

  • Scaleability is Paramount: The paper strongly argues that any effective data center network benchmark must be scaleable, not just in terms of size, but also in its ability to adapt to different applications, users, and underlying technologies.
  • Application-Driven is Key: Abstract traffic models are insufficient. The real-world behavior of applications is the most critical factor in shaping network traffic patterns.
  • Emulation Bridges the Gap: Network emulation, when done correctly, can accurately mimic the behavior of physical data center networks, making large-scale testing feasible.
  • Micro vs. Component Traces: The distinction between micro and component-level traces is important for capturing both specific traffic phenomena (like bursts) and the overall behavior of an application.
  • Beyond Rated Specs: Switch chip evaluations reveal that while rated parameters (like exchange capacity) are indicative, they don't tell the whole story. Actual performance under realistic traffic patterns can vary significantly, suggesting a need to choose devices based on specific traffic profiles.

Key Results

  • Reality Verification: Experiments show that emulated network traffic closely matches physical network traffic in terms of time-series (CDF) and spatial (traffic matrix) similarity for representative big data applications.
  • Scaleability Demonstration: DCNetBench successfully generates diverse micro and component-level traces by varying applications and network topologies, demonstrating its ability to scale.
  • Switch Chip Performance: The evaluation of five switch chips using micro traces revealed that the HUAWEI S7706 consistently performed optimally across various metrics, while other switches showed varying performance profiles, highlighting the importance of matching device capabilities to traffic patterns.

Limitations and Future Work

  • Emulation Overhead: While effective, emulation inherently introduces some overhead compared to physical hardware, though the paper demonstrates this impact is minimal for their chosen metrics.
  • Trace Diversity: While twelve micro and four component traces are generated, the authors suggest that further expansion with more diverse applications and scenarios would enhance the benchmark's coverage.
  • Future Work: The authors implicitly suggest that expanding the library of applications and network configurations within DCNetBench would be a natural next step. Further research could also explore more sophisticated methods for trace analysis and classification.

Summary updated

BibTeX
@misc{liu2023dcnetbench,
  author = {Ke Liu and Wanling Gao and Chunjie Luo and Cheng Huang and Chunxin Lan and Zhenxing Zhang and Lei Wang and Xiwen He and Nan Li and Jianfeng Zhan},
  title = {{DCNetBench: Scaleable Data Center Network Benchmarking}},
  year = {2023},
  eprint = {2302.11866},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2302.11866},
}

Loading related content...

Figure