Rate-Based Transport Protocols for High-Latency Networks

24 concepts12 papers1 proposal

When moving massive datasets across transcontinental distances, traditional networking protocols hit a performance wall. This bottleneck is not caused by a lack of physical bandwidth, but by a fundamental mismatch between traditional congestion control algorithms and high Bandwidth-Delay Product (BDP) networks.

To unlock the full capacity of modern fiber-optic wide-area networks (WANs), a paradigm shift was required: moving away from the kernel-level, window-based congestion control of TCP toward application-level, rate-based transport protocols built on top of UDP.

The High-BDP Bottleneck of Traditional TCP

To understand why traditional methods fail, we must look at how TCP manages data flow. TCP relies on window-based congestion control, specifically the Additive Increase Multiplicative Decrease (AIMD) algorithm. The sender maintains a "congestion window" that dictates how many packets can be in flight before requiring an acknowledgment (ACK) from the receiver. The time it takes for an ACK to return is determined by the network's Round Trip Time (RTT).

In a local area network, RTT is negligible. But on a transcontinental or transoceanic link, RTT can easily exceed 100 milliseconds. When a network has both high bandwidth (e.g., 1 Gbps or higher) and high latency, its BDP is massive—meaning megabytes of data must be "in flight" simultaneously to keep the pipe full.

TCP struggles catastrophically in these environments due to two core behaviors:

  1. Sensitivity to Packet Loss: TCP treats any dropped packet as a definitive signal of network congestion. On long-distance links, minor physical layer instabilities or random packet drops cause TCP to instantly slash its congestion window in half.
  2. Slow Recovery Tied to RTT: After slashing its window, TCP increases its window size linearly, adding only a small amount of data per RTT. On a high-latency link, it can take minutes of error-free transmission for TCP to ramp back up to full capacity.

If another packet drops during this ramp-up period, the throughput collapses permanently, leaving expensive gigabit links severely underutilized.

Shifting Control to the Application Layer via UDP

To bypass these limitations without requiring unrealistic, global upgrades to operating system kernels, researchers decoupled the transport logic from the OS network stack. By utilizing UDP—a raw, connectionless protocol that does not enforce windowing, ordering, or flow control—developers gained a blank canvas to implement custom reliability mechanisms directly in user space.

As noted by Yunhong Gu et al. in Experimental studies of data transport and data access of earth-science data over networks with high bandwidth delay products, building on top of UDP allows protocols to remain highly efficient and "TCP-friendly" while operating entirely within the application layer. This architectural flexibility led to the development of UDP-based Data Transfer (UDT).

In UDT: UDP-based data transfer for high-speed wide area networks, Yunhong Gu et al. demonstrated that this framework could achieve near-full utilization of available links, reaching speeds of ~940 Mbps on a 1 Gbps link with 110ms RTT, whereas standard TCP reached only ~128 Mbps under identical conditions. To ensure compatibility with legacy systems, the authors also designed middleware gateways capable of transparently bridging standard TCP/HTTP applications with high-performance UDT streams internally (Yunhong Gu et al., Data mining middleware for wide-area high-performance networks).

The Mechanics of Rate-Based Congestion Control

The core breakthrough that allows these protocols to conquer high latency is the transition from window-based to rate-based congestion control.

Instead of tying the transmission of new packets directly to the arrival of individual ACKs, a rate-based protocol injects packets into the network at a continuous, precisely calculated interval (the packet sending rate). As explained by Robert L. Grossman et al. in Data mining using high performance data clouds: experimental studies using sector and sphere, bypassing the window mechanism allows the protocol to saturate high-speed links immediately, completely independent of the network's RTT.

Window-Based (TCP):
Sender ---> [Packet 1, 2, 3] ---> Receiver
Sender <--- Wait for ACKs (1 RTT) <--- Receiver (Throughput bound by RTT)

Rate-Based (UDP/UDT):
Sender ---> [Packet Rate: 1000 pps] ---> Receiver (Continuous Stream)
Sender <--- Periodic Control Signals <--- Receiver (Throughput independent of RTT)

This RTT independence is achieved by decoupling data transmission from control signaling. In Teraflows over Gigabit WANs with UDT, Yunhong Gu et al. detailed how UDT uses periodic control packets to monitor packet arrival intervals and error rates at the destination. Rather than slashing throughput when a single drop occurs, the protocol uses these metrics to dynamically fine-tune the packet injection rate.

By decoupling the data plane from the control plane, as highlighted by Chung‐Uk Lee et al. in DATA TRANSFER TEST FOR KMTNet DATA, the protocol isolates the data stream from the latency penalty of the feedback loop, allowing it to remain highly resilient against random packet loss over transcontinental distances.

Enabling Wide-Area Distributed Computing

By mitigating the network latency penalty, rate-based UDP transport fundamentally changes how geographically distributed computing systems are designed. It allows wide-area networks to be treated as a high-performance system back-plane, effectively making remote data centers perform as if they were local (Robert L. Grossman et al., Compute and Storage Clouds Using Wide Area High Performance Networks).

This protocol foundation enabled the creation of large-scale scientific data clouds:

Ultimately, by replacing window-based feedback loops with precise, application-level rate control, these protocols transformed high-latency WANs from a severe operational bottleneck into high-throughput pipelines capable of supporting real-time global scientific collaboration.

Key Foundational Work

Teraflows over Gigabit WANs with UDT (2005) — contributed 3 concept elements to this direction.
Experimental studies of data transport and data access of earth-science data over networks with high bandwidth delay products (2004) — contributed 2 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_udp_based_data_transfer_de71987c,
  title = {UDP-based data transfer},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/udp-based-data-transfer/rdr_de71987c612448ffbf98063eee00a8b9},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_udp_based_data_transfer_de71987c_v1,
  title = {UDP-based data transfer},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/udp-based-data-transfer/rdr_de71987c612448ffbf98063eee00a8b9/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake