Towards "Zero-buffer" Datacenter Networks

Mowei Wang, Cong Liang, Yashe Liu, Yong Jiang, Yong Cui

Abstract

In this paper, we investigate the possibility of building a data center network (DCN) with simplified zero-buffer switches. It exposes a stringent requirement to the traffic control scheme to ensure perfect collision-free between packets. We find it is infeasible, if not impossible, to make a large-scale DCN completely zero buffering. We take a step back and propose Zeropod, a new data center architecture that enables zero buffering in pod scale, which can be expanded with buffered devices. With time synchronization, Zeropod avoids collisions by performing fine-grained centralized control on data packets and coordination-free scheduling on control packets. It leverages an optimistic sending mechanism to reduce the latency overhead induced by the centralized control. Our preliminary simulation results indicate that Zeropod successfully meets the requirement of collision-free in the zero-buffer pod and maintains similar performance with conventional buffered DCN. It follows the trend of simplifying the network by pushing the functionalities to the edge. Zeropod explores an extreme end of the design spectrum and can be treated as a thought experiment to raise discussion in the community of data center networks.

1. Introduction

Modern datacenters face a growing disparity between exploding traffic volumes and the capacity of switching chips. The physical area of these chips is increasingly dominated by Serializer/Deserializer (SerDes) logic, leaving limited space for on-chip buffers. Traditionally, these buffers are essential for managing bursty traffic and preventing packet loss. However, they now consume up to 30% of chip area and introduce latency issues like "bufferbloat."

Zeropod explores a radical alternative: removing packet buffers entirely from switches within a "pod" and replacing them with a centralized scheduler. By reallocating the saved silicon area to additional SerDes, we can potentially double switching capacity.

2. System Design

Zeropod adopts a "dumb network, smart edges" philosophy through several key mechanisms:

2.1 Zero-Buffer Pods and Centralized Scheduling

Within a network pod, all Top-of-Rack (ToR) and Aggregate switches are bufferless. To prevent collisions, a centralized per-pod scheduler coordinates host transmissions. This scheduler manages host-level collision avoidance, while endhost SmartNICs handle flow-level prioritization.

Figure 1: The Zeropod architecture, featuring zero-buffer pods coordinated by a centralized scheduler and connected via buffered core switches.

2.2 Hierarchical Scaling and Optimistic Sending

To scale beyond a single pod, Zeropod utilizes traditional buffered switches at the Core layer. These cores act as asynchronous barriers, isolating the timing requirements of individual pods. To mitigate the scheduling delay for small, latency-sensitive "mice flows," Zeropod employs an "optimistic sending" mechanism where these flows are sent on random paths without prior scheduling, accepting potential drops in exchange for lower average latency.

3. Evaluation

We evaluated Zeropod using the YAPS packet-level simulator across standard workloads including WebSearch, Incast, and collective communication patterns (AllReduce, AlltoAll).

3.1 Performance Results

The primary advantage of Zeropod is the ability to trade buffer area for bandwidth. A "Zeropod (2x)" configuration, utilizing the saved chip area to double port density, significantly outperforms traditional buffered networks.

Figure 4: Comparison of Goodput and 99th-percentile Mice flow FCT between traditional DCNs and Zeropod configurations.

As shown in Figure 4, Zeropod (2x) maintains superior goodput even at 100% load. Furthermore, it achieves a ~50% reduction in 99th-percentile Flow Completion Time (FCT) for mice flows compared to traditional buffered DCNs using DCTCP. In heavy incast scenarios, Zeropod’s deterministic scheduling allows for near-ideal finish times, avoiding the buffer overflows that plague traditional switches.

4. Discussion and Conclusion

While Zeropod demonstrates significant performance gains, it introduces dependencies on precise time synchronization and centralized scheduler availability. Future work will focus on the robustness of the scheduler and the integration of this architecture with emerging bufferless optical switching technologies. By pushing network functionality to the edge, Zeropod offers a compelling path toward higher-capacity, lower-latency datacenter fabrics.

Summary updated

Abstract

In this paper, we investigate the possibility of building a data center network (DCN) with simplified zero-buffer switches. It exposes a stringent requirement to the traffic control scheme to ensure perfect collision-free between packets. We find it is infeasible, if not impossible, to make a large-scale DCN completely zero buffering. We take a step back and propose Fastpod, a new data center architecture that enables zero buffering in pod scale, which can be expanded with buffered devices. With time synchronization, Fastpod avoids collisions by performing fine-grained centralized control on data packets and coordination-free scheduling on control packets. It leverages an optimistic sending mechanism to reduce the latency overhead induced by the centralized control. Our preliminary simulation results indicate that Fastpod successfully meets the requirement of collision-free in the zero-buffer pod and maintains similar performance with conventional buffered DCN. It follows the trend of simplifying the network by pushing the functionalities to the edge. Fastpod explores an extreme end of the design spectrum and can be treated as a thought experiment to raise discussion in the community of data center networks.
BibTeX
@misc{wang2021towards,
  author = {Mowei Wang and Cong Liang and Yashe Liu and Yong Jiang and Yong Cui},
  title = {{Towards "Zero-buffer" Datacenter Networks}},
  year = {2021},
  eprint = {2109.13065},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2109.13065},
}

Loading related content...

Figure