Introduction
Traditional data center networks (DCNs) are reaching a scaling limit. As bandwidth demands push toward 400Gbps and beyond, the electrical switches that form the backbone of these networks are becoming prohibitively expensive and power-hungry. Optical Circuit Switches (OCS) offer a promising alternative due to their high bandwidth and energy efficiency. However, OCS hardware suffers from significant reconfiguration latency—typically around 30ms—which is far too slow to react to microsecond-scale traffic bursts.
Existing solutions generally fall into two categories: traffic-aware designs that attempt to predict and rapidly reconfigure (often failing due to complexity) and traffic-agnostic designs that rotate through fixed patterns. The latter often incurs a heavy "bandwidth tax" by forcing traffic onto inefficient multi-hop paths via Valiant Load Balancing (VLB). This paper introduces TROD (Threshold Routing based Optical Datacenter), a novel architecture that leverages the "weak stability" of aggregate traffic to provide a high-performance, low-complexity optical DCN.
System Architecture
TROD is designed as a PoD-level (Point of Delivery) architecture. While individual server traffic is highly volatile, the aggregate traffic between PoDs—which may contain hundreds of servers—exhibits recurring patterns over longer timescales.
As shown in the architecture above, the Core Layer utilizes OCS to interconnect PoDs. TROD employs a Topology Engineering (ToE) approach where the optical layer is reconfigured infrequently (e.g., daily). Using a Progressive Filling Algorithm, TROD builds a topology that prioritizes direct links for heavy, recurring traffic pairs while maintaining a residual uniform mesh to handle unexpected bursts.
Threshold-Based Routing (tVLB)
The core innovation of TROD is its routing scheme, Threshold-based VLB (tVLB). Unlike traditional VLB, which spreads all traffic across indirect paths, tVLB prioritizes efficiency. Traffic is sent via the shortest direct path as long as the flow rate remains below a pre-defined threshold ($p$). Only the excess traffic from a burst is load-balanced across two-hop indirect paths.
A key advantage of TROD is its compatibility with off-the-shelf hardware. As illustrated in the pipeline diagram, tVLB can be implemented on standard OpenFlow 1.3 or P4 switches (such as the Barefoot Tofino). It utilizes standard meter tables to monitor flow rates and DSCP remarking to distinguish between direct-path traffic and excess burst traffic that must be routed through the Group Table to indirect paths.
Key Results and Performance
The researchers evaluated TROD using real-world production traces from Facebook and synthetic bursty traces. The results demonstrate significant improvements over both traditional and state-of-the-art optical designs:
- Flow Completion Time (FCT): TROD reduced FCT by 1.15–2.16× compared to Google’s Jupiter DCN and by at least 2× compared to other optical designs like Helios or Rotornet.
- Comparison to Clos Networks: TROD outperformed the industry-standard non-blocking 1:1 Clos network when the optical layer was over-provisioned by a factor of just 1.2.
- Implementation Accuracy: Physical hardware tests on a P4 switch confirmed that the tVLB rate-limiting mechanism is highly precise, with an error rate of less than 0.3%.
Conclusion
TROD demonstrates that by combining low-frequency topology reconfiguration with a clever threshold-based routing scheme, it is possible to overcome the reconfiguration latency of optical switches. By eliminating the "bandwidth tax" for the majority of traffic and utilizing standard hardware features, TROD provides a practical and efficient path forward for next-generation optical data centers.
Summary updated