A Quantitative Analysis and Guidelines of Data Streaming Accelerator in Modern Intel Xeon Scalable Processors

Reese Kuper, Ipoom Jeong, Yifan Yuan, Jiayu Hu, Ren Wang, Narayan Ranganathan, Nam Sung Kim

Abstract

As semiconductor power density is no longer constant with the technology process scaling down, modern CPUs are integrating capable data accelerators on chip, aiming to improve performance and efficiency for a wide range of applications and usages. One such accelerator is the Intel Data Streaming Accelerator (DSA) introduced in Intel 4th Generation Xeon Scalable CPUs (Sapphire Rapids). DSA targets data movement operations in memory that are common sources of overhead in datacenter workloads and infrastructure. In addition, it becomes much more versatile by supporting a wider range of operations on streaming data, such as CRC32 calculations, delta record creation/merging, and data integrity field (DIF) operations. This paper sets out to introduce the latest features supported by DSA, deep-dive into its versatility, and analyze its throughput benefits through a comprehensive evaluation. Along with the analysis of its characteristics, and the rich software ecosystem of DSA, we summarize several insights and guidelines for the programmer to make the most out of DSA, and use an in-depth case study of DPDK Vhost to demonstrate how these guidelines benefit a real application.

Introduction: Addressing the "Datacenter Tax"

In modern cloud environments, CPUs frequently expend significant resources—often exceeding 30% of total cycles—on "housekeeping" tasks such as memory copies (memcpy), buffer zeroing, and checksum calculations (CRC32). This phenomenon, known as the "datacenter tax," diverts processing power away from primary application logic. While previous offloading technologies like I/OAT and CBDMA existed, they were limited by high initialization overhead and memory pinning requirements. The Intel® Data Streaming Accelerator (DSA) in 4th Gen Xeon processors aims to overcome these hurdles with a more versatile and low-overhead architecture.

DSA Architecture and Software Ecosystem

The DSA architecture is designed for high-performance data movement and transformation. It consists of multiple groups, each containing work queues (WQs) and processing engines (PEs). Software interacts with DSA through memory-mapped portals, allowing for low-latency work submission without kernel intervention in the data path.

Intel DSA hardware and software stack overview.

The software ecosystem includes the idxd kernel driver and user-space libraries like libaccel-config and accel-config for device discovery and configuration. This stack enables seamless integration into high-level frameworks such as DPDK, SPDK, and Libfabric.

Evaluation Methodology

The study utilizes a two-pronged approach:

  1. Microbenchmarking: Using dsa-perf-micros to isolate hardware characteristics like transfer size impact, batching efficiency, and performance across memory tiers (DRAM, remote NUMA, and CXL-attached memory).
  2. Real-World Integration: Evaluating end-to-end performance in critical software stacks, including DPDK Vhost for networking, Libfabric for HPC/ML, CacheLib for web caching, and SPDK for storage.

Key Insights and Programming Guidelines

  • The 4KB Threshold: For synchronous offloading, the overhead of descriptor setup makes DSA beneficial primarily for transfers larger than 4KB. However, asynchronous offloading can yield gains for transfers as small as 256 bytes by allowing the CPU to perform other work in parallel.
  • Cache Pollution Mitigation: DSA can bypass the CPU cache, writing data directly to memory. This prevents transient data from evicting "hot" application data from the Last Level Cache (LLC).
  • Optimal Batching: A "sweet spot" of 4–8 descriptors per batch is recommended for synchronous tasks to balance submission overhead reduction against completion latency.
  • CXL Synergy: DSA is highly effective at managing data movement between standard DRAM and higher-latency CXL-attached memory.

Performance Results

DSA demonstrates substantial improvements over previous generations and software-only implementations. It achieves an average of 2.1x greater throughput than CBDMA.

Throughput improvement across various data operations and transfer sizes.

Specific application gains include:

  • Networking: DPDK Vhost packet forwarding rates improved by 1.14x to 2.29x.
  • HPC/ML: Large message transfers via Libfabric saw up to a 5.1x speedup, contributing to an 8.8% reduction in BERT training time.
  • Storage: SPDK reached line-rate saturation with fewer CPU cores by offloading CRC32 tasks.

Limitations and Future Work

Despite its strengths, DSA throughput typically saturates at approximately 30 GB/s due to internal I/O fabric constraints. Additionally, the "startup cost" remains a challenge for very small operations, and achieving peak performance often requires manual refactoring for asynchronous execution. Future research will explore more complex in-flight transformations and DSA's role in disaggregated memory architectures.

Summary updated

Abstract

As semiconductor power density is no longer constant with the technology process scaling down, modern CPUs are integrating capable data accelerators on chip, aiming to improve performance and efficiency for a wide range of applications and usages. One such accelerator is the Intel Data Streaming Accelerator (DSA) introduced in Intel 4th Generation Xeon Scalable CPUs (Sapphire Rapids). DSA targets data movement operations in memory that are common sources of overhead in datacenter workloads and infrastructure. In addition, it becomes much more versatile by supporting a wider range of operations on streaming data, such as CRC32 calculations, delta record creation/merging, and data integrity field (DIF) operations. This paper sets out to introduce the latest features supported by DSA, deep-dive into its versatility, and analyze its throughput benefits through a comprehensive evaluation. Along with the analysis of its characteristics, and the rich software ecosystem of DSA, we summarize several insights and guidelines for the programmer to make the most out of DSA, and use an in-depth case study of DPDK Vhost to demonstrate how these guidelines benefit a real application.
BibTeX
@misc{kuper2023quantitative,
  author = {Reese Kuper and Ipoom Jeong and Yifan Yuan and Jiayu Hu and Ren Wang and Narayan Ranganathan and Nam Sung Kim},
  title = {{A Quantitative Analysis and Guidelines of Data Streaming Accelerator in Modern Intel Xeon Scalable Processors}},
  year = {2023},
  eprint = {2305.02480},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2305.02480},
}
Figure