TL;DR
As Compute Express Link (CXL) memory moves from theory to reality, this paper provides the first comprehensive evaluation of true, commercially available CXL hardware. The authors reveal that common emulation techniques (using remote NUMA nodes) fail to capture real CXL performance characteristics, particularly regarding cache coherence and cache hierarchy interactions. To exploit these findings, they introduce Caption, a dynamic OS page allocation policy that improves the throughput of memory-bandwidth-intensive applications by up to 24%.
What Problem Does This Paper Address?
Modern data centers face a severe "memory wall." Increasing memory capacity and bandwidth using traditional DDR interfaces is hitting physical limits due to CPU pin constraints and high power consumption. Compute Express Link (CXL) has emerged as a open standard to solve this, allowing systems to dynamically expand memory capacity and bandwidth over high-speed serial PCIe links.
Because genuine CXL hardware has been notoriously difficult to acquire, the research community has relied heavily on emulation—specifically, treating a remote CPU socket's memory in a multi-socket system (a remote NUMA node) as a proxy for CXL memory. This paper addresses a critical question: Does emulated CXL memory actually behave like true CXL memory? If not, prior research conclusions and operating system policies might be fundamentally flawed.
Methods
The authors evaluate a true CXL-ready system featuring a 4th-generation Intel Xeon (Sapphire Rapids) processor connected to three distinct, genuine CXL memory devices from different manufacturers:
- CXL-A: Hard ASIC IP with DDR5-4800 memory.
- CXL-B: Hard ASIC IP with DDR4-2400 memory.
- CXL-C: Soft FPGA IP with DDR4-3200 memory.
To characterize these devices, they use Intel's Memory Latency Checker (MLC) alongside a custom-built microbenchmark called memo. Unlike standard tools, memo isolates the latency and bandwidth of specific x86 instruction types, such as temporal loads/stores (ld/st) and non-temporal loads/stores (nt-ld/nt-st) which bypass the CPU cache.
Using insights from this hardware characterization, the authors design Caption, an OS-level dynamic page allocation policy. Caption periodically samples hardware performance counters (like L1 miss latency and instructions per cycle) using Intel Performance Counter Monitor (PCM). It feeds these metrics into a lightweight linear regression model to predict system throughput and dynamically adjusts the ratio of pages allocated to local DDR vs. CXL memory using a greedy optimization algorithm.
Datasets
The paper evaluates system and application performance across a diverse suite of workloads:
- Latency-Sensitive Applications: Redis (benchmarked via the Yahoo! Cloud Serving Benchmark, YCSB), DeathStarBench (an open-source social network microservice framework), and FIO (flexible I/O tester evaluating OS page cache performance).
- Throughput-Intensive Applications: Facebook’s Deep Learning Recommendation Model (DLRM) embedding reduction step, and the four highest memory-bandwidth-demanding benchmarks from the SPECrate CPU2017 suite (fotonik3d, mcf, roms, and cactuBSSN).
Key Insights and Ideas
- CXL Memory $\neq$ Remote NUMA Memory: True CXL memory lacks remote CPU cores and caches. Consequently, the local CPU utilizes an on-chip hardware structure for fast cache coherence checks rather than traversing slow inter-socket interconnects (like Intel UPI). This allows true CXL to achieve up to 26% lower latency and significantly higher bandwidth efficiency for parallel memory accesses compared to emulated setups.
- Breaking LLC Isolation: In Sub-NUMA Clustering (SNC) mode, the CPU normally isolates Last Level Cache (LLC) slices to prevent applications from interfering with one another. However, the authors discovered that accessing CXL memory breaks this isolation: evicted L2 cache lines from CXL memory can be distributed across all LLC slices in any SNC node. This effectively grants CXL-accessing applications a 2–4× larger LLC capacity, mitigating CXL's latency penalties for cache-friendly workloads.
- Naïve Allocation is Harmful: Blindly splitting pages 50/50 between local DDR and CXL memory (the default OS behavior) degrades performance. Microsecond-scale applications like Redis experience tail-latency spikes of 10–82%. Even bandwidth-hungry applications can suffer lower throughput due to memory controller contention if allocations aren't dynamically balanced based on the specific CXL device's capabilities.
Key Results
- Latency Variations: Hardware controller design matters immensely. The ASIC-based CXL-A device exhibited only 35% longer load latency than local DDR5, whereas the FPGA-based CXL-C device suffered nearly 3× longer latency (see Figure 3).
- Caption Policy Performance: When evaluating mixed throughput workloads (SPEC-Mix), Caption successfully navigated the trade-offs between DDR and CXL bandwidth. It automatically converged on optimal allocation ratios (allocating 29%–41% of pages to CXL), outperforming the best static OS allocation policies by up to 24%.
- Page Cache Viability: Allocating the Linux page cache to CXL memory using FIO resulted in a mere ~3% latency increase for 4 KB blocks, proving that CXL is an excellent, cost-effective candidate for OS caching layers.
Limitations and Future Work
While Caption demonstrates excellent results, its predictive linear regression model was primarily trained on weight values derived from DLRM profiles. The authors note that accuracy could be further improved by training the model on a wider, more diverse set of applications. Additionally, current hardware limitations prevent the OS from directly reading performance counters located on the CXL devices themselves (such as device-side queue lengths or exact link utilization). Accessing these device-native statistics remains an open avenue for building even more responsive tiered-memory managers.
Summary updated