On the performance of commodity hardware for low latency and low jitter packet processing

Charalampos Stylianopoulos, Magnus Almgren, Olaf Landsiedel, Marina Papatriantafilou, Trevor Neish, Linus Gillander, Bengt Johansson, Staffan Bonnier

Introduction

The evolution of industrial networking toward Industry 4.0 and 5G requires "Ultra-Reliable Low Latency Communication" (URLLC). Traditionally, these requirements necessitated specialized, expensive hardware. However, the industry is shifting toward Virtual Network Functions (VNFs)—software-based networking running on standard commodity servers. While this transition offers flexibility and cost savings, standard servers are typically optimized for high throughput rather than the deterministic low latency required by factory robotics or autonomous vehicles. This paper investigates the feasibility of using off-the-shelf hardware for time-critical industrial applications by identifying and mitigating sources of latency and jitter.

Methodology

The researchers utilized a testbed consisting of two 20-core Intel Xeon servers connected via 10 Gb Ethernet. To evaluate performance, they compared three distinct packet-processing architectures:

  1. Kernel-based (Linux Bridge): The standard Linux networking stack.
  2. User-space (Standard DPDK): Utilizing the Data Plane Development Kit (DPDK) to bypass the kernel, though typically employing packet buffering to maximize throughput.
  3. Latency-Optimized DPDK: A modified DPDK implementation where packet buffering was disabled to ensure immediate forwarding.

To further minimize jitter, several system-level "hardening" techniques were applied, including thread isolation (dedicating CPU cores to network tasks), interrupt management, and disabling Intel Turbo Boost to maintain a constant CPU frequency. Traffic was generated using MoonGen, sending 64-byte UDP packets at a rate of 100,000 packets per second, with hardware-level timestamping providing 100-nanosecond precision.

Results and Analysis

The experimental results demonstrate that standard high-performance frameworks like DPDK, while efficient for throughput, introduce significant latency at low traffic rates due to batch processing. By disabling buffering, the researchers achieved a dramatic shift in the latency profile.

Latency distribution across different processing methods

As shown in the latency distribution, the latency-optimized DPDK approach (red) achieved an average latency of 5.7 $\mu$s, representing a 9.8x improvement over standard DPDK (56.1 $\mu$s) and significantly outperforming the Linux bridge (16.8 $\mu$s). Furthermore, system-level optimizations reduced average jitter by 2.4x, bringing maximum jitter down from 12.52 $\mu$s to 4.80 $\mu$s.

However, these gains in latency and predictability come at the cost of maximum throughput. The "empty pipe" strategy—optimizing for zero buffering—limits the system's ability to handle high-volume bursts.

Latency vs. Throughput trade-off for optimized stacks

As illustrated in the throughput analysis, the latency-optimized version maintained sub-10 $\mu$s latency until approximately 4.3 Gbps, at which point the system became overwhelmed. In contrast, throughput-optimized configurations successfully reached the 10 Gbps line rate but at the expense of much higher baseline latency.

Conclusion

This study proves that commodity hardware can meet the stringent requirements of URLLC if properly configured. By prioritizing predictability over peak speed and disabling throughput-oriented features like buffering and dynamic frequency scaling, researchers achieved near-deterministic performance suitable for industrial IoT. Future research will focus on the overhead introduced by complex VNFs and virtualization layers like Docker or KVM, as well as the integration of these wired optimizations with 5G wireless segments.

Summary updated

Abstract

With the introduction of Virtual Network Functions (VNF), network processing is no longer done solely on special purpose hardware. Instead, deploying network functions on commodity servers increases flexibility and has been proven effective for many network applications. However, new industrial applications and the Internet of Things (IoT) call for event-based systems and midleware that can deliver ultra-low and predictable latency, which present a challenge for the packet processing infrastructure they are deployed on. In this industry experience paper, we take a hands-on look on the performance of network functions on commodity servers to determine the feasibility of using them in existing and future latency-critical event-based applications. We identify sources of significant latency (delays in packet processing and forwarding) and jitter (variation in latency) and we propose application- and system-level improvements for removing or keeping them within required limits. Our results show that network functions that are highly optimized for throughput perform sub-optimally under the very different requirements set by latency-critical applications, compared to latency-optimized versions that have up to 9.8X lower latency. We also show that hardware-aware, system-level configurations, such as disabling frequency scaling technologies, greatly reduce jitter by up 2.4X and lead to more predictable latency.
BibTeX
@inproceedings{stylianopoulos2020performance,
  author = {Charalampos Stylianopoulos and Magnus Almgren and Olaf Landsiedel and Marina Papatriantafilou and Trevor Neish and Linus Gillander and Bengt Johansson and Staffan Bonnier},
  title = {{On the performance of commodity hardware for low latency and low jitter packet processing}},
  booktitle = {DEBS},
  year = {2020},
  doi = {10.1145/3401025.3403591},
}
Figure