Scalable and Efficient Diagnosis for 5G Data Center Network Traffic

Chi Harold Liu, Jun FanIEEE Access 2014 · OpenReview

Abstract

Data center networks (DCNs) for 5G are expected to support a large number of bandwidth-hungry applications with exploding data volumes. Identifying the cause of link congestion between switch ports is critical for maintaining performance. This paper proposes two sketch-based algorithms, α-conservative update (α-CU) and P(d)-CU, to diagnose network congestion in real-time. By leveraging the inherent skewness of different network services, these methods achieve high estimation accuracy and low computational complexity. Experimental results on real DCN traces demonstrate a 50% reduction in estimation error and significant improvements in scalability compared to existing techniques.

The Challenge of 5G Network Monitoring

As 5G networks scale, DCNs must handle a massive influx of traffic from diverse applications. Identifying the specific "flow"—a sequence of packets between a source and destination—causing link congestion is a significant challenge. Traditional monitoring tools like NetFlow or sFlow rely on packet sampling, which often misses bursty "elephant flows"—the small fraction of high-volume flows responsible for the majority of network congestion.

Typical 5G Data Center Network (DCN) architecture showing core, aggregation, and access layers.

To address this, researchers use compact data structures called sketches to summarize traffic. However, existing sketches face a "trilemma": they struggle to simultaneously provide high accuracy, low computational complexity, and high scalability. For instance, the popular Count-Min (CM) sketch is fast but prone to overestimation, while the Conservative Update (CU) sketch is more accurate but computationally expensive for high-speed 5G ports.

Proposed Sketch-Based Algorithms

The authors propose two enhancements to the Conservative Update (CU) approach to balance performance and accuracy:

  1. α-Conservative Update (α-CU): This probabilistic hybrid allows operators to tune the balance between speed and accuracy. For every update, the algorithm decides with probability α to perform a CU update and with probability 1−α to perform a faster CM update.
  2. Partitioned CU (P(d)-CU): This is the paper's primary innovation. Instead of a single large sketch, the authors partition the sketch vertically into sub-sketches dedicated to specific traffic types (e.g., HTTP, DNS). This prevents "mice" flows from one service from colliding with and corrupting the "elephant" counts of another.
  3. Real-time Moving Average: To handle sliding windows, they implement an exponential discounting factor (γ). This allows the system to maintain a real-time view of traffic without the memory overhead of storing multiple time-sliced sketches.

Experimental Evaluation and Insights

The study evaluates these algorithms using a real-world DCN trace from a commercial airline booking service, comprising over 29.6 million records. Analysis of this data confirmed that DCN traffic follows Zipf’s Law, where a tiny fraction of flows accounts for the vast majority of traffic volume.

Workload distribution analysis (PDF and CDF) for the entire DCN and individual switches, confirming the heavy-tailed nature of the traffic.

Key Results

  • Accuracy: P(d)-CU reduced the Average Relative Error (ARE) by at least 50% compared to the standard CU algorithm.
  • Speed: P(d)-CU demonstrated an 18% reduction in compute time compared to CU, bringing it closer to the speed of the less accurate Count-Min sketch.
  • Recall: All proposed algorithms achieved over 95% recall, successfully identifying the top elephant flows even with limited memory.
  • Scalability: The memory cost remains fixed regardless of the number of flows, making the solution ideal for high-density 5G environments.

Conclusion

By incorporating service-aware partitioning and probabilistic updates, α-CU and P(d)-CU provide a scalable and efficient solution for real-time congestion diagnosis. The ability to isolate traffic types based on their skewness parameters allows for significantly higher accuracy without the computational penalties typically associated with high-fidelity network monitoring.

Summary updated

Abstract

Data center networks (DCNs) for 5G are expected to support a large number of different bandwidth-hungry applications with exploding data, such as real-time search and data analysis. As a result, significant challenges are imposed to identify the cause of link congestion between any pair of switch ports that may severely damage the overall network performance. Generally, it is expected that the granularity of the flow monitoring to diagnose network congestion in 5G DCNs needs to be down to the flow level on a physical port of a switch in real time with high-estimation accuracy, low-computational complexity, and good scalability. In this paper, motivated by a comprehensive study of a real DCN trace, we propose two sketch-based algorithms, called α-conservative update (CU) and P(d)-CU, based on the existing CU approach. α-CU adds no extra implementation cost to the traditional CU, but successfully trades off the achieved error with time complexity. P(d)-CU fully considers the amount of skew for different types of network services to aggregate traffic statistics of each type of network traffic at an individual, horizontally partitioned sketch. We also introduce a way to produce the real-time moving average of the reported results. By theoretical analysis and sufficient experimental results on a real DCN trace, we extensively evaluate the proposed and existing algorithms on their error performance, recall, space cost, and time complexity.
BibTeX
@inproceedings{liu2014scalable,
  author = {Chi Harold Liu and Jun Fan},
  title = {{Scalable and Efficient Diagnosis for 5G Data Center Network Traffic}},
  booktitle = {IEEE Access},
  year = {2014},
  url = {https://openreview.net/forum?id=f-J4d2LdLL},
}
Figure