To maximize network throughput and hardware utilization, modern networks increasingly rely on parallelism. Whether aggregating multiple physical links, distributing traffic across asymmetric paths, or processing packets across parallel pipelines, spreading data out is the most effective way to bypass single-link bottlenecks.
However, this parallelism introduces a fundamental challenge: packet reordering. When packets from the same flow traverse different paths or processing branches, variations in latency cause them to arrive out of order.
Historically, out-of-order delivery was treated as a severe error. Traditional transport protocols like TCP interpret out-of-order packets as a sign of network congestion and packet loss, triggering unnecessary retransmissions. High-performance protocols like Remote Direct Memory Access (RDMA) strictly require in-order delivery to function, making standard multi-path load balancing unusable. To avoid this, hardware designers built complex, expensive reordering buffers at network boundaries, creating a significant latency and silicon overhead.
To resolve this tension, recent research explores three core strategies: enforcing in-order delivery through smart adaptive routing, designing transport protocols that are resilient to reordering, and using mathematical coding to reconstruct data when physical order is completely lost.
Order-Preserving Adaptive Routing
Instead of forcing packets down a single static path to maintain order, can we route traffic adaptively while guaranteeing that packets never overtake one another?
Path Clearing and Flowcuts
A intuitive solution to this problem is to track the state of paths dynamically. In Flowcut Switching: High-Performance Adaptive Routing with In-Order Delivery Guarantees, Tommaso Bonato proposes a deterministic path-switching mechanism. Instead of routing packets randomly or probabilistically, the system tracks in-flight data. A network flow is only reassigned to a new path once all previously transmitted packets have completely cleared the original path. By ensuring the old path is drained, packets sent on the subsequent path can never overtake older data, providing a strict in-order guarantee without requiring any destination buffering.
In-Order Flow Deflection
In datacenter networks where congestion happens rapidly, waiting for a path to drain can introduce queueing delays. To protect latency-sensitive short flows, Balajee Vamanan introduces In-Order Flow Deflection (IOFD) in Dart: Divide and Specialize for Fast Response to Congestion in RDMA-Based Datacenter Networks. IOFD allows switches to deflect packets onto alternate paths to bypass localized congestion. Crucially, the hardware mechanism deflects entire bursts or manages the switching logic such that it preserves packet order, maintaining seamless compatibility with strict protocols like RDMA.
Deterministic Segmentation
In on-chip networks (NoCs), avoiding out-of-order delivery is vital for maintaining cache coherence without massive hardware overhead. C.X Yang et al. address this in IDFT: An Intermediate Node Based Deterministic Fault Tolerant Routing in 2D Mesh. To bypass hardware faults without triggering the out-of-order delivery typical of adaptive routing, the authors split a packet's journey into multiple segments. Each segment is governed by strict, deterministic dimension-ordered routing. This hybrid approach allows the network to route around failures adaptively while ensuring that packets arrive predictably and in sequence.
Designing Resilient Transport Protocols
An alternative philosophy is to stop enforcing order at the network layer altogether. If the endpoints of a network can handle out-of-order packets gracefully, core switches can become simpler, cheaper, and faster.
Time-Based Loss Detection
Traditional TCP implementations infer packet loss when they receive a specific number of duplicate acknowledgments (ACKs). If packets arrive out of order, they trigger false duplicate ACKs, causing the sender to mistakenly slow down and retransmit data.
As highlighted in the IEEE International Conference on High Performance Switching and Routing (IEEE HPSR 2023), modern TCP implementations have evolved to decouple loss detection from sequence numbers. Algorithms like Recent Acknowledgment (RACK) use time-based offsets rather than packet counts to detect loss. This resilience allows networks to deploy highly efficient, load-balanced switch architectures that freely reorder packets without degrading transport-layer performance.
"Any Next Packet" Request Models
In large-scale data dissemination and Information-Centric Networking (ICN), enforcing strict sequential delivery can cause severe head-of-line blocking; if a single early packet is delayed, the receiver cannot process any subsequent data.
To solve this, Jiachen Chen et al. propose an "Any Next Packet" request model in SAID: A Control Protocol for Scalable and Adaptive Information Dissemination in ICN. Instead of demanding packets in a rigid sequence, receivers accept any available data packet that is immediately present in the network. By filling missing gaps through localized repairs later, the protocol fully utilizes available multi-path network capacity and avoids the performance collapse typical of traditional in-order constraints.
Index-Based Coding for Shuffled Channels
The most extreme manifestation of out-of-order delivery occurs in channels where physical sequence is completely destroyed, such as DNA-based data storage or highly unstable wireless broadcast networks. In a DNA storage system, millions of synthesized DNA strands are mixed together in a fluid; when sequenced, they are retrieved in a completely randomized, shuffled order.
To reconstruct data from a completely unordered set of fragments, researchers rely on information theory and coding. Work by Reinhard Heckel et al. in Fundamental Limits of DNA Storage Systems and Ilan Shomorony et al. in Capacity Results for the Noisy Shuffling Channel demonstrates the mathematical optimality of a coded-index strategy.
Instead of relying on the physical medium to maintain order, a portion of each data packet or DNA strand is dedicated to a unique address tag or index. The authors prove that prefixing fragments with these identifiers allows the decoder to mathematically reassemble the global message with zero error, even when the channel introduces both noise and complete physical shuffling. This shifts the burden of ordering entirely from the physical network layer to the logical information layer.
Key Foundational Work
Preferential Treatment of Acknowledgment Packets in a Differentiated Services Network (2001) — contributed 3 concept elements to this direction.